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Bringing Research to Life reveals research in the trenches. 


Much of research activity isn’t obvious or visible. These opening vignettes are designed to take the student 
behind the door marked RESEARCH. Through the activities of the principals at Henry & Associates, students 
learn about research projects, many that were revealed to the authors off the record. The characters and names 


of companies are fictional, but the research activities they describe are real—and happening behind the scenes in 


hundreds of firms every day. 


. . Lv 
bringingresearchtolife 


“Based on the poll results that are on your screen, you 
have reached a strong consensus on your first priority. 
The research strongly supports that you should be 
er courier contracts to address the 


negotiating stro 
in-transit damage issues. Congratulations,” concluded 
Jason. 

“That wraps up our briefing, today. Sara and I are 
happy to respond to any e-mail questions any of you 
might have after reading the summary report that has 
been delivered to your e-mail. Our e-mail address is on 
screen, and it is also on the cover of the report. Myra, 
I'm handing control of the meeting back to you.” 

As Myra 
holding up a s 


ted to conclude the meeting, Sara was 


n in front of Jason that read. “Turn off 
your microphone.” Jason gave a thumbs-up sign and 
clicked off his mic. 

“Thank you, Jason,” stated Myra. “The research 
has clarified some critical issues for us and you have 
helped us focus on some probable solutions. This 
concludes the meeting. I'll be following up soon with 
an e-mail that contains a link to the recorded archive 
of this presentation, allowing you to share it with your 
staff. You will also be asked to participate in a brief 
survey when you close the Web-presentation window. 
Td really appreciate your taking the three minutes it 
will take to complete the survey. Thank you all for 
attending.” 

As soon as the audience audio was disconnected, 
Myra indicated, “That went well, Jason. The use of 
the Q&A tool to obtain their pre-report ideas for action 
was a stroke of genius. When you posted the results as 
anal] and had them indicate their first priority, they 


chapter 2 


learningobjectives 
After reading this chapter, you should understand. . . 
What issues are covered in research ethics. 


The goal of “no harm” for all research activities and what constitutes “no harm” for participant, researcher, 
and research sponsor. 


The differing ethical dilemmas and responsibilities of researchers, sponsors, and research assistants. 


The role of ethical codes of conduct in professional associations. 


all over the board. It helped them understand that 


one purpose of the research and today’s meeting was to 
bring them all together.” 

“Sara gets the credit for that stroke of genius,” 
claimed Jason after removing his microphone and 
clicking on his speakerphone. “She is a strong 
proponent of interaction in our briefings. And she 
continually invents new ways to get people involved 
and keep them engaged.” 


“Kudos, Sar claimed Myra. “Who gets the 


credit for simplifying the monthly comparison chart?” 

“Those honors actually go to our intern, Sammye 
Grayson,” shared Sara. “I told her while it was a 
suitable graph for the written report; it was much too 
complex a visual for the presentation. She did a great 


job. I'll pass on your praise.” 


“Well,” asked Myra, “where do we go from here?” 


“Jason and I will field any questions for the next 


from you or your staff,” explained Sara. “Then 
we will consider this project complete—until you 


contact us again. 
“About thi 
e-mail from Jean-Claude. He wants to meet with you 


” Myra paused, “I’ve just received an 


both about a new project he has in mind. He asks if he 
could pick you up at the Boca airport on Friday, about 
2:30 p.m. He says his flying office will have you back 


in time for an early dinner. 
Sara consulted her iPhone and indicated she was 
available. Jason looked at his own calendar and smiled 
across the desk at Sara. “Tell Jean-Claude we'll meet 
him at the airport. Any idea what this new project is 
about?” 
“Not a clue!” 


Learning Objectives 
serve as memory flags. 


Learning objectives serve as a road map as students 
start their journey into the chapter. Read first, these 
objectives subconsciously encourage students to 


seek relevant material, definitions, and exhibits. 


Special tools for today’s visual learner. 


A transformation is taking place in many of our classrooms. During the last decade, more and more of our 


students have become visual—not verbal—learners. Verbal learners learn primarily from reading text. Visual 


learners need pictures, diagrams, and graphs to clarify and reinforce what the text relates. 


Integrated research process exhibits reveal a rich 
and complex process in an understandable way. 


Every textbook has exhibits. We use these tables and line 
drawings to bring key concepts to life and make complex 
concepts more understandable. 


Within our array of exhibits is a very special series of 
32 fully integrated research process exhibits. Each 
exhibit in this series shares symbols, shapes, and colors 


with others in the series. 


>Exhibit 1-4 The Research Process 


Exploration Exploration 


Research Design 
Strategy 
(type, purpose, time frame, scope, environment) 


Sampling 
Design 


Data Collection 
Design 


Exhibit 1-3 is the overview exhibit of the research 
process, to which all other exhibits related to the process 
will link. 


Subsequent exhibits (like this one for survey design) 


Data Collection & Preparation 


Data Analysis & Interpretation 


Research Reporting 


show more detail in a part of this process. 


Another exhibit in the series might layer the main process 
exhibit with additional information (like this exhibit from 


the ethics chapter). 


>Exhibit 2-1 Ethical Issues and the Research Process 


<— * Sponsor nondisclosure 


Exploration Exploration 


* Sponsor's right to quality 
research 
* Sponsor's right of purpose 
nondisclosure 
* Researcher's right to absence 
of sponsor coercion 
* Researcher's right to absence 
L. of sponsor deception 
Research Design 
Strategy 
(type, purpose, time frame, scope, environment) 


* Participant deception 
* Sponsor's right to quality 
L research 


Data Collection Sampling 


Design 


* Participant's right of informed 
consent 

* Participant's right to privacy 
(refusal) 

* Sponsor's right to quality 
research 

* Researcher's right to absence 

L_ of sponsor coercion 


* Sponsor's right to quality 
researc! 

* Participant's right to privacy 

* Participant deception 

* Sponsor's right to sponsor 
nondisclosure 

|_* Researcher's right to safety 


Data Analysis & Interpretation 


Research Reporting 


* Sponsor's right to findings 
nondisclosure 

* Participant's right to 
confidentiality 

* Sponsor's right to quality 
research 

* Researcher's right to absence 

L_ of sponsor coercion 


Management 
Decision 


Management 
Decision 


Chapters 2-5 


Appendix A 


Chapters 6-14 


Chapter 15 


Chapters 16-18 


Chapters 19-20 


>Exhibit 13-5 Flowchart for Instrument Design: Phase 2 


Questions 


Classification 
Qu 


Demagraphic | 
Eeanomie | 
Sociological | 
Geographic | 


Instrument 
Development 


Pretest Individual 
Questions 


Some topics deserve more 
attention—with their own chapter! 


An emphasis on presentation. 


Increasingly, researchers are making oral presentations of 
their findings though Web-driven technologies. We address 
this and other oral presentation formats and issues with a 


separate chapter. 


>chapter 20 


Presenting Insights and Findings: 
Oral Presentations 


>learningobjectives 


After reading this chapter, you should understand . . . 


4. How the oral research presentation and is similar to traditional public speaking, 


2 Why histor 


rhetorical theory has p 


Vluence on business presentation skils in the 21st century. 
3 How to plan for the research presentation, 
4 The frameworks and pattems of organizing a presentation, 


5 The 


‘and dlferences between the types of materials designed to support your points. 


ese an and, what needs to be assembled and checked 
jon and venue are ready, 


8 Why pract nt to 
to be certain that arrangements for the occasi 


All researchers increasingly >chapter 7 
need qualitative skills. Qualitative Research 


> 
Researchers increasingly admit that quantitative research 
: 2 >learningobjectives 
can’t reveal all they need to know to make smart business forsramoenme rae 
decisions. We capture the best of the current qualitative ore seers seers eee 
methods and reveal where and how they are used. 4: Te vey of quate research methods 


>chapter 5 


Clarifying the Research Question through 


management d i lem ma to Secondary Data and Exploration 
resea re h d es i g n rT] >learningobjectives 


> After reading this chapter, you should understand... 


Help in moving from 


This is where talented people can steer research in the 


wrong or right direction. We devote a chapter to 


providing students with a methodology for making the S heysaiia eeu ak elation ema nebo 


11 The purposes and process of exploratory research. 


2 ‘Two types and three levels of management dex 


n-related secondary sources, 


al factors for evaluating the value of a. id Its content. 


stand the management cl 
ch question (and, uitimately, investigative questions ar 


ugh the stages 
1d measurement 


right decisions more often. 


>chapter 2 


Ethics in Business Research 


Ethical issues get the 
attention they deserve. 


eT >learningobjectives 
Ethical issues abound in business research but may 


go unnoticed by students who need a framework to _ Meo an tree a avd ante Yona rey e 
discuss and understand these issues. We devote a | IDE Xe Oa ae REEL 


chapter to building that framework. 


Today, it would be remiss to say that the privacy 
profession is anything but flourishing. Companies 
are increasingly hiring privacy officers and even 
elevating them to C-suite positions; the European 
Commission has proposed a statute in its amended 
data protection framework that would require data 


protection officers at certain organizations, and at 
the International Association of Privacy Professionals 
(IAPP) membership recently hit 10,000 worldwide. 

An IS, 


Alter reading this chapter, you should understand. . 


1 What issues are covered in research ethies. 


International Associatio 


The Challenges and Solutions to Mobile Questionnaire Design 


“As researchers, we need to be sensitive to the unique chal- 
lenges respondents face when completing surveys on mo- 
bile devices,” shared Kristin Luck, CEO of Decipher. “Small 
screens, inflexible device-specific user input methods, and 
potentially slow data transfer speeds all combine to make 
the survey completion process more difficult than on a typi: 
cal computer. Couple those hindrances with reduced atten- 
tion spans and a lower frustration threshold and it's clear that, 


Please indicate your gender: 


as researchers, we must be proactive in the design of both ste 


the questionnaire and user-interface in order to accommodate 
mobile respondents and provide them with an excellent survey 
experience.” 

Decipher researchers follow key guidelines when designing 
surveys for mobile devices like smart phones and tablets. 


Which of the following age groups do you fall into? 


* Ask 10 or fewer questions 
* Minimize page refreshes—longer wait times reduce 
participation. 
* Ask few questions per page—many mobile devices 10 of 24 
have limited memory. 


* Use simple question modes—to minimize scrolling 
Keep question and answer text short—due to smaller 
screens 


If unavoidable, limit scrolling to one dimension (vertical 
is better than horizontal). 

Use single-response or multiple-response radio button 
or checkbox questions rather than multidimension grid 
questions. 

Limit open-end questions—to minimize typing. 

Keep answer options to a short list. 

For necessary longer answer-list options, use drop- 
down box (but limit these as they require more clicks to 
answer). 


* Debate use of progress bar—it may encourage 
completion but also may require scrolling. 


* Minimize distraction 
¢ Use simple, high-contrast color schemes—phones 
have limited color palettes. 
© Minimize JavaScript due to bandwidth concerns. 


© Eliminate Flash on surveys—due to incompatibility with 
iPhone. 


Luck is passionate about making sure that researchers recog- 
nize the special requirements of designing for mobile as mobile 
surveys grow in use and projected use, S shares her expertise at 


* Minimize all non-essential content 
* If used, limit logos to the first or last survey page. 
* Limit privacy policy to first or last survey page. 


be asked of participants. Four questions, covering numerous issues, guide the instrument designer in 
selecting appropriate question content: 

° Should this question be asked (does it match the study objective)? 

° Is the question of proper scope and coverage? 

* Can the participant adequately answer this question as asked? 


° Will the participant willingly answer this question as asked? 


Students learn by and 
deserve the best examples. 


Snapshots are 
research examples 
from the researcher’s 
perspective. 


conferences worldwide. www.decipherinc.com =|] 


Snapshots are like mini-cases: They help a 
student understand a concept in the text by 
giving a current example. As mini-cases 

they are perfect for lively class discussion. 
Each one focuses on a particular application of 
the research process as it applies to a particular 
firm and project. You’ll find more than 

82 of these timely research examples 
throughout the text and more in the Instructor’s 
Manual. 

Web addresses speed secondary data searches 


on companies involved with the example. 


| __ Icons help students link parts of a richer, more complex 


example, told over a series of chapters. 


®) Some examples are so rich in detail that one Snapshot or exhibit just isn’t sufficient. MindWriter is a 


MindWriter | computer laptop manufacturer that prides itself on customer service, especially when it comes to laptop 


repair at its CompleteCare center. Each time you see this icon in the text, you'll be learning more about the 


customer satisfaction research that Henry & Associates is doing. 


The Closeup offers a more in-depth examination of a 


key example. 


Sometimes you just need more time and space to showcase all the detail of an example. This glimpse 


of the Closeup from Chapter 16 reveals two pages from a discussion on tabular data. 


Using Tables to Understand Data 


Because the researcher's primary job is to discover the mes- 
sage revealed by the data, he or she needs every tool to reveal 
the message. Authors Sally Bigwood and Melissa Spore in their 
book Presenting Numbers, Tables, and Charts suggest that the 
table is the ultimate tool for extracting knowledge from data. 


The presence of any number within a table is for comparison 
with a similar number—from last year, from another candidate, 
from another machine, against a goal, and so forth. Using the 
author's rules for table creation, a researcher exploring data by 
‘constructing a table should: 


* Round numbers. Rounded numbers can be most easily compared, er 


or relationship of one number to another. 


jing us to more easily determine the ratio 


* If precision is critical to the number (e.g. you are researching taxes or design 
drug interactions), don't round the numbers. 


* Arrange the num- * Order numbers from largest to smallest number, 
bers to reveal * Ina vertically arranged table, order the largest number at the top. 
patterns. * In a horizontal arrangement, order the largest numbers on the left. 


* When looking for changes over time, order the numbers by year, from most distant (left or top) to 
most recent. 


© An average pro\ point for compari 
* Don't use an average if the raw data reveal a bimodal 


distribution. 


* Totals emphasize the big picture. 


* Percentages show proportionate relat han raw data 


ships more eas! 


* Compare like scales | * 
ina single table. 


Convert numbers to a common scale when the numbers reflect different scales (e.g., grams 
Versus ounces of cereal consumption; monthly salary data versus hourly wage data). 


* Choose simplicity © Se 
over complexity é 
= Com fata, 
* Use empty space * Design a table with a smaller number of columns than rows. 
and design to guide |. gi 
Rese enines Single-space numbers that must be compared. 
that must be com- | * Use gridiines to group numbers within a table; avoid gridlines between numbers that must 
pared and to make be compared. 
patterns and excep- |, Z 
elas Use empty space to create gutters between numbers in simple tables. 
* Right-align column headers and table numbers, 
+ Summarize each * Write a phrase or sentence that summarizes your interpretation of the data presented; don't 
data d leave interpretation to chance. 


used as the tith 
n any numbers. 


© Summar 5 might t 
* The summary nead not mentic 


at 


or chart in the final 


arch report. 


* Label and title . 
tables for clarity of 
message. 


Titles should be comprehensive: Include what (subject of the tile or message), where: (f data 
have a geographic base), when (date or time period covered), and unit of measure. 


* Include common information in the title: It lengthens a title but shortens the table's column 
headings. 
* Avoid abbreviations in column headings unless well known by your audience, 


* Avoid footnotes; if used, use symbols—like the asterisk—rather than numbers (numbers used as 
footnotes can be confused with the content numbers of the table). 


© For reference, provide an undertable source line for later reference. 


>closeupcont’d 


‘AN EXAMPLE 


‘Assume you were adetermining whether to expand info westem Europe with distribution faciities to service online purchases of your 
specialty goods company. 


Table 1 Spending by Internet Users in Selected Western 
European Countries 2010 (EUROs in Billions) 


Annual Annual 
Spending Purchases 

France Euro 664.5 16 

Germany Euro 658.0 

Italy Euro 345.5 14 

Spain ro 560.1 40 

United Kingdom Euro 2284.9 36 


We start with the above table that presents data developed from several studies on online shopping and purchasing behavior in 
selected countries in western Europe. The data are ordered alphabetically by country, While arranging in alphabetical order may be 
ideal for randomization or reduction of bias, It isn't a logical choice for clarity of data presentation. 

What data might you need to help you make your decision about distribution facilities? Do you need to know the average 
transaction size? If you don’t know the conversion rate of the euro to the dollar, can you interpret the table? Should you put 
your investment in the United Kingdom or elsewhere? 


Table 2 E5 Per Capita One-Year Online Spending (2010) 


Annual Spending ‘Average Annual Annual Spending 


(EUROs) Purchases (uss) 

United Kingdom 2284.9 36 1736.2 
Germany 658.0 20 500.0 
France 664.5 16 505.0 
414 262.6 

10 425.6 


Table 2 recasts the data using Bigwood and Spore's guidelines. First the table tile has changed; now the annual period on which 
the spending data are based is more obvious, as well as the fact that we are looking at spending per capita for the top 6 European 
Union performers, known as the E5. We've also changed the column headers to reflect currency, and we have right-justified the 
headers and the numbers. We've rearranged the table by Average Spending (EURO) in descending order and interpreted the (EURO) 
‘column by adding a dollar conversion column, We might not need the rightmost column if we were euro spenders ourselves but, if we 
‘are more: familiar with another currency, the addition of this column helps us interpret the data, With this arrangement, does Germany 
look attractive? While it might not currently appear to be as strong a contender as the United Kingdom, we know it is fiscally strong 
and located in a more central location to the other countries being considered. 


PicProfile offers a memory 
visual to enhance an 
example. 


In research, as in life, sometimes a picture is worth 


more than words. Sometimes you need to see what 
is being described to fully understand the 


foundation research principle. 


Social Media Analytics 


Online Communities 


Mobile Surveys [I 


Text Analytics [im 


Webcam-based 
Interviews [> 


Apps-based Research 
Eye Tracking 

Mobile Ethnography |i 
Mobile Qualitative [fi 
Virtual Environments 
Crowdsourcing 
Visualization Analytics 
Prediction Markets 


Biometric Response |i 


NeuroMarketing 


Gamification Methods 125 ll Research provider (n=669) 
il Research client (n=149) 
Facial Analysis 
T T T T 1 
30 40 50 60 70 


>picprofile 


According to the 2012 Greenbook Research Industry Trends (GRIT) report, the top four emerging techniques, among both 
research buyers and providers all involve Internet use. “A big climber, from actual 2011 to expected 2012, is Mobile Surveys, 
with clients/buyers jumping from a current 17% to an expected 53% and vendors expecting the increase to be from 24% to 
64%.” Some speculate that the mobile survey may be approaching its tipping point. Other methodologies, like Mobile Qualita- 
tive, Mobile Ethnography, and Gamification, are getting a lot of buzz in the industry, but have yet to capture buyer/client sup- 
port to the same degree that they have earned researcher interest. As in previous studies, researcher interest tends to lead on 
methodology. http://www.greenbook.org/PDFs/GRIT-S12-Full.pdf 


Source: “Spring 2012 Greenbook Research Trends Report,” GreenBook® | New York AMA Communication Services Inc., February 2012, p. 22. 
Leonard Murphy, “GRIT Sneak Peek: What Emerging Research Techniques Will Be Used in 2012?” Greenbook, posted February 20, 2012. 
Downloaded April 18, 2012, http://www.greenbookblog.org/201 2/02/20/grit-sneak-peek-what-emerging-research-techniques-will-be- 
used-in-2012/. 


Learning aids cement the concepts. 


Discussion questions that go one 
step further. 


Five types of discussion questions reveal differing levels of 


understanding—from knowing a definition to applying a concept. 


>discussionquestions 


Terms in Review 


1 


x 


° 


rs 


Making Research Decisions 


How does qualitative research differ from quantitative 5 Assume you are a manufactur 
research? like Hamitton Beach/Proctor Silex, and you wi 
How do data from qualitative research differ from data in termine if some innovative designs with unusual shapes 


Sor 


leveloped for the European market could be 
july marketed in the U.S. market. What qualitative 


ch? 


quantitative res 


Why do senior executives feel more comfortable relying on } 
research would you recommend, and why? 

quantitative data than qualitative data? How might a qual 

alive rese: any lessen the senior-level executive's 6 NCR Corporation, known as a world leader in ATMs 

skepticism? - poi le (POS) retail checkout scar and check- 
in kiosks at airports, announced in June 2009 that it 

Distinguish between structured, semistructured, and un- Ls 


would move 


headquarters from Dayton (OH) 


structured interviews. 


>keyterms 


bibliography 98 
data marts 102 index 98 
data mining 102 
data warehouse 102 
dictionary 98 
directory 100 


encyclopedia 98 


expert intervie 


exploratory research 94 


=discussionquestions 


Terms in Review 


1. Explain how each of th 


© Authority 


d_ Audience 


handbook 99 
individual depth interview (ID!) 94 
investigative questions 112 
literature search 94 


measurement questions 118 << 


predesigned 118 


factors for a second- 


making value. 


mail survey a relatively low-cost self-administered study both 
delivered and returned via mail 

main effect the average direct influence that a particular treat 
‘ment of the IV has on the DV independent of other factors, 

management dilemma the problem or opportunity that requires 
‘a decision; a symptom of a problem or an early indication of 
an opportunity 

management question the management dilemma rest 
question format 
eration and 
control of a 

management r 


ed as “choice of objectives.” 


1 of solutions,” or “troubleshooting or 
ort a report written for the nontechnically oi 


ented manager or client 

management-research question hierarchy process of sequen: 
tial question formulation that leads a manager or researcher 

. questions, 

manuscript reading the verbatim reading of a fully written 
presentation, 

mapping rules a scheme for assigning numbers to aspects of an 
‘empirical event. 

marginal(s) a term for the column and row totals in a 
cross-tabulation, 


from management dilemma to measurem« 


matching process analogous to quota sampling for assigning 
participants to experimental and control groups by having 
participants match every descriptive characteristic used in the 
research; used when random assignment is not possible; an 
attempt to eliminate the effect of confounding variables that 
‘group participants so that the confounding variable is present 


mean the arithmetic average of a data distribution, 

mean square the variance computed as an average or mean. 

measurement assigning numbers to empirical events in com: 
pliance with a mapping rule 

‘measurement questions the questions asked of the participants 
or the observations that must be recorded. 

measures of location term for measure of central tendency in a 
distribution of data; see also central tendency 

measures of shape statistics that describe departures from the sym: 
metry of a distribution; ak.a. moments, skewness, and kurtosis 

measures of spread statistics that describe how scores cluster 
or scatter in a distribution; aka, dispersion or variability 


(variance, standard deviation, range, interquartile range, and 


mini-group a group interview involving two to six people. 

missing data information that is missing about a participant or 
data record: should be discovered and rectified during data 
preparation phase of analysis; e.g., miscoded data, out-of 
range data, or extreme values, 

mode the most frequently occurring value in a dats distribution: 
data may haye more than one mode 

model a representation of a system that is constructed to study 

at system or the system as a whole. 

je (MV) a second independent variable, be. 

nificant contributory or contingent effect 

lly stated IV-DV relationship, 

moderator a trained interviewer used for group interviews such 


as focus groups, 


if «classification of data collection that includes ob: 
servation studies and data mining of organizational databases. 
motivated sequence a presentation planning approach that in: 
volves the ordering af ideas to follow the normal processes 
of human thinking; motivates an audience to respond to the 


presenter's purpose 
multicollinearity occurs when more than two independent vaci 
ables are highly correlated. 

al scale a scale that seeks to simultaneously 
fe more than one attribute of the participant or object. 

ensional scaling (MDS) a scaling technique to simul 
taneously measure more than one attribute of the participant 
or object; results are usually mapped; dev 
picture or map of the locations of som 
others on various dimensions or properties; especially useful 


relative to 


multiphase sampling sce double sampling, 

multiple-choice, multiple-response scale a scale that offers 
the participant multiple options and solicits one or more an. 
swers (nominal or ordinal data); aka. checklist. 

multiple-choice question a measurement question that offers 


wgle-response scale a scale that poses more 
responses but seeks a single answer, or one 


that seeks a single rating from a gradation of preference, in. 
terest, or agreement (nominal or ordinal data); a.k.a. nudtiple 
choice question. 

multiple comparison tests compare group means following the 
finding of a statistically significant F test. 


primary sources 96 
research question(s) 112 
secondary sources 96 


source evaluation 


tertiary sources 97 


‘management question 108 


custom-designed 118 
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Key terms indexed at the end of the 
chapter and defined in the glossary. 


-——— Glossary reinforces the importance 


of learning the language of 


research. 


Supplements offer the tools 
students and faculty ask 
for...and more. 


>Abstract 


www.troy! 


and what makes television advertising most effective 


and develop the subsequent ad campaign. www.marcusthomasllc. 


‘Troy-Bilt works with Marcus ‘Thomas LLC to develop marketing communications, 
including television advertising campaigns. As part of its creative development, 
Marcus ‘Thomas needed to understand what motivated the yardwork enthusiast, 
ithin the lawn and garden 
industry. Marcus Thomas developed a hypothesis that “consumers who are in the 
market to purchase a product process television advertising differently than those 
who are not in the market for lawn and garden equipment.” It used an online 
survey employing a control group, with embedded video, to test thi 


hypothesis 


On the book’s Online Learning Center (www.mhhe.com/ 
cooper! 2e), students will find cases (like this new one) and 
data sets, a research proposal, a sample student project, and 
supplemental material for several chapters, including 
templates for charting data, how the research industry works, 
bibliographic databases searching tips, complex experimental 
designs, test markets, pretesting, and multivariate analysis. 
You'll also find 34 cases, nine of which are full video cases. 


Also, several written cases have video components included. 


For undergraduate students just learning about research 
methods or graduate students advancing their research 
knowledge, each new edition of Business Research Meth- 
ods promises—and has continually delivered—not only a 
teachable textbook but a valued reference for the future. 
As a mark of its worldwide acceptance as an industry 
standard, Business Research Methods is available in nine 
international editions and four languages. 

When you are creating a 12th edition, you don’t want to 
tinker too much with what has made instructors adopt your 
textbook for their students or what has prompted research- 
ers to use it as a valuable shelf reference. But to ignore 
change in the research environment would be negligent. 


Leading 


We used the 2012 GreenBook Research Industry Trends 
(GRIT) Report as a starting point for creating the 12th edi- 
tion. This large study of research suppliers and research 
clients gave us clear direction on emerging techniques and 
how the research field was changing. We focused our ef- 
forts on obtaining examples of these changes and they are 
included in content throughout the book and in Snapshots 
and PicProfiles—both contentwise and visually. 


Responsive . . . to Students 
and Faculty 


Snapshots, PicProfiles, and CloseUps are the way we re- 
veal what is timely and current in research. We wait until 
such issues are more mainstream before giving the topic 
a permanent place within the text. In fact, of the 82 Snap- 
shots and PicProfiles featured, 35 are completely new and 
one-half of the CloseUps had major updates. Of these new 
examples, you will find topics dealing with biometrics, 
eye tracking via the Web, mobile surveys, online com- 
munities, listening tours, location-based tracking, talent 
analytics, incentivizing participants, data visualization, 
mixed mode surveys, mixed access recruiting, charting, as 
well as Internet research, cloud computing, using Excel 
in data analysis and presentation, Smartphone research, 
dirty data, gut hunches, wildcat surveys, and more. And 
you'll discover research stories that relate to such organi- 
zations or brands as Mercedes-Benz, TNS-Infratest, NTT 
Communications, Next Generation Market Research, In- 
teractive Advertising Bureau, Groupon, TrustE, Decipher, 
Living Social, Troy-Bilt, among numerous others. 

There are currently about 200 images and text art sup- 
porting our learning objectives; you will discover that 
over one-quarter are new to this edition. We’ve updated 
our “From the Headlines” discussion questions, covering 
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product introductions, employee issues, legal proceed- 
ings, advertising campaigns, and many more topics and 
added more research examples to the Instructor’s Manual, 
for use in class discussions or testing. 

Our book is designed for a one-semester course although 
under no circumstance is it imagined that the entire book 
be covered. In an effort to make the book more user 
friendly yet give faculty members tremendous flexibility in 
choosing materials for the theme they set for their course, 
we have created an Online Learning Center for the text. 
Analogous to cloud computing, we stored regularly used 
data on McGraw-Hill’s servers that can be easily accessed 
through the Internet. Central to that design, we moved ma- 
terial from chapter appendices to the Online Learning Cen- 
ter thereby reducing the physical size of the book that our 
own students often carry with them. Among those items 
available at the Online Learning Center are How the In- 
dustry Works, Bibliographic Database Searches, Advanced 
Bibliographic Searches, Complex Experimental Designs, 
Test Markets, and Pretesting Options and Discoveries. 
Since many research methods courses for undergraduates 
don’t use multivariate statistics, we’ve moved our chap- 
ter “Multivariate Analysis: An Overview” to the Online 
Learning Center for the benefit of graduate students. You’ ll 
also find written and video cases, questionnaires, data sets, 
a sample student project, and digital support materials re- 
lated to some of our Snapshots and CloseUps in the Online 
Learning Center. 

We continue to use chapter and end-of-text appendices 
for information that, given the differing skills and knowl- 
edge of their students, instructors may want to emphasize. 
We retained end-of-chapter appendices related to Craft- 
ing Effective Measurement Questions and Determining 
Sample Size, as well as end-of book appendices related 
to a sample proposal, a focus group guide, non-parametric 
statistics, and statistical tables. 


Fine-Tuned 


Process Series of Exhibits The core pedagogy of 
Business Research Methods is based on an understanding 
that student learners are of three types: visual, auditory, 
and kinesthetic. These exhibits offer a detailed, graphical 
map of the research process or a more detailed breakout 
of each subprocess, perfect for hands-on projects. Each 
of these exhibits is linked to others in the series with a 
consistent use of shape and color. You'll find 32 of these 
exhibits throughout the text. Changes in process exhibits, 
other exhibits, and embedded tables resulted in twenty- 
three major modifications using new information, data, or 
graphs throughout the text. 


Online Learning Center = There is a wealth of informa- 
tion, samples, templates, and more in this Web depository. 


Written Cases. Cases offer an opportunity to tell re- 
search stories in more depth and detail. You'll find a 
new case, Marcus Thomas LLC Tests Hypothesis for 
Troy-Bilt Creative Development, complete with its 
online questionnaire, at the Online Learning Center. 
You'll also find cases about hospital services, lotter- 
ies, data mining, fundraising, new promotions, and 
website design, among other topics, featuring orga- 
nizations like Akron Children’s Hospital, Kelly Blue 
Book, Starbucks, Yahoo!, the American Red Cross, 
and more. 


Video Cases. We are pleased to continue to make 
available a first in video supplements, several short 
segments drawn from a two-hour metaphor elicitation 
technique (MET) interview. These segments should 
be invaluable in teaching students to conduct almost 
any type of individual depth interview and to explain 
the concept of researcher—participant rapport. Four of 
our video cases were written and produced especially 
to match the research process model in this text and 
feature noted companies: Lexus, Starbucks, Wirthlin 
Worldwide (now Harris Interactive), Robert Wood 
Johnson Foundation, GMMB, Visa, Bank One, Team 
One Advertising, U.S. Tennis Association, Vigilante 
New York, and the Taylor Group. 


Web Exercises. It is appropriate to do Web searches 
as part of a research methods course, so each chapter 
offers one or more exercises to stimulate your 

students to hone their searching skills. Due to the 
ever-changing nature of Web URLs, however, we offer 
these exercises in the Instructor’s Manual. 


Articles, Samples, and Templates. Students often 
need to see how professionals do things to really 
understand, so you’ll find a sample EyeTrackShop 
report, a Nielsen report of using U.S. Census data, an 
Excel template for generating sample data displays, 
and more. 


Sample Student Project. Visualization of the fin- 
ished deliverable is crucial to creating a strong re- 
search report. 


Collaborative 


When revising an edition, many individuals and compa- 
nies contribute. Here are some who deserve special recog- 
nition and our gratitude. 


¢ To all those researchers and consultancy profes- 
sionals who shared their projects, images, ideas, 
perspectives, and the love of what they do through 
e-mails and interviews and who helped us develop 
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cases, Snapshots, PicProfiles, exhibits, or CloseUps, 
or provided new visuals, we extend our heartfelt ap- 
preciation: Edwige Winans, Marcus Thomas, LLC; 
Jennifer Hirt-Marchand, Marcus Thomas, LLC; 
Kristin Luck, Decipher; Tom H. C. Anderson, An- 
derson Analytics; Leonard F. Murphy, GreenBook; 
Rachel Sockut, Innerscope; Erica Cenci, Brady PR 
for OpinionLab; Olescia Hanson, The Container 
Store; Cynthia Clark, 1tol Magazine; Rachel Soc- 
kut, Innerscope; Betty Adamou, Research Through 
Gaming Ltd.; Debra Semans, Polaris Marketing 
Research; Keith Chrzan, Maritz Research, Inc.; 
Michael Kemery, Maritz Research, Inc.; Christian 
Bauer, Daimler AG; Kai Blask, TNS Infratest; 
Melinda Gardner, Novation; Pete Cape, SSI; Keith 
Phillips, SSI; Sean Case, Research for Good; Nels 
Wroe; SHL; Ephraim (Jeff ) Bander, EyeTrack- 
Shop; Ron Sellers, Grey Matter Research & Con- 
sulting; Guadalupe Pagalday, Qualvu.com; Sandra 
Klaunzler, TNS Infratest; Betty Adamou, Research 
Through Gaming Ltd; Steve August, Revelation; 
Kathy Miller, GMI (Global Market Insite, Inc.); 
Takayuki NOZOE, NTT Communications Cor- 
poration; Janeen Hazel, Luth Research; Christine 
Stricker, RealtyTrac; Stephanie Blakely, The Pros- 
per Foundation; Jennifer Frighetto, Nielsen; Andy 
Pitched, Research Triangle Institute (RTI Interna- 
tional); Jeffrey C. Adler, Centric DC Marketing Re- 
search; Josh Mendelssohn, Chadwick Martin Bailey, 
Inc.; Ruth Stan at, SIS International Research; Sha- 
ron Starr, IPC, Inc.; Lance Jones, Keynote Systems; 
Keith Crosley, Proofpoint; Christopher Schultheiss, 
SuperLetter.com; Hy Mariampolski, QualiData 
Research Inc; Julie Grabarkewitz and Paul Herrera, 
American Heart Association; Holly Ripans, Ameri- 
can Red Cross; Mike Bordner and Ajay Gupta, 
Bank One; Laurie Laurant Smith, Arielle Burgess, 
Jill Grech, David Lockwood, and Arthur Miller, 
Campbell-Ewald; Francie Turk, Consumer Con- 
nections; Tom Krouse, Donatos Pizza; Annie Burns 
and Aimee Seagal, GMMB; Laura Light and Steve 
Struhl, Harris Interactive; Emil Vicale, Herobuild- 
ers.com; Adrian Chiu, NetConversions; Eric Lipp, 
Open Doors Organization; Stuart Schear, Robert 
Wood Johnson Foundation; Elaine Arkin, consultant 
to RWJF; Colette Courtion, Starbucks; Mark Miller, 
Team One Advertising; Rebecca Conway, The Tay- 
lor Research Group; Scott Staniar, United States 
Tennis Association; Danny Robinson, Vigilante; 
Maury Giles, Wirthlin Worldwide; and Ken Mallon, 
Yahoo!; and colleagues at IBM and Lenovo. 


To Jane Ducham, our Developmental Editor and 
Christina Kouvelis, Managing Developmental 

Editor, who facilitated the complex process and 
to our Senior Brand Manager Thomas Hayward, 
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who felt strongly enough about us as successful 
authors to support this revision. 

To the remainder of our McGraw-Hill team, for 
making the book a priority: 

Content Project Manager: Mary Jane Lampe 
Marketing Manager: Heather Kazakoff 

Media Project Manager: Prashanthi Nadipalli 
Buyer: Nichole Birkenholz 

Designer: Studio Montage 

Photo Researcher: Danny Meldung 


e To our faculty reviewers for their insights, sugges- 
tions, disagreements, and challenges that encour- 
aged us to look at our content in different ways: 
Robert Wheatley, Troy University; Gary Tucker 
Northwestern, Oklahoma State University; War- 
ren Matthews, LeTourmeau University; Marjolijn 
Vandervelde, Davemport University; Ron E. Holm; 
Cardinal Stritch University (Director of Distance 
Learning); Erika Matulich, University of Tampa; 
Cheryl O’ Meara Brown, University of West Geor- 
gia; Kay Braguglia, Hampton University; Ken 
Zula, Keystone College; Bob Folden, Texas A&M 
University; Scott Baker, Champlain College; Scott 
Bailey, Troy University; Robert Balik, Western 
Michigan University—Kalamazoo; John A. Ballard, 
College of Mount St. Joseph; Jayanta Bandyopad- 
hyay, Central Michigan University; Larry Banks, 
University of Phoenix; Caroll M. Belew, New Mex- 
ico Highlands University; Jim Brodzinski, College 
of Mount St. Joseph; Taggert Brooks, University of 
Wisconsin—La Crosse; L. Jay Burks, Lincoln Uni- 
versity; Marcia Carter, University of Southern New 
Hampshire; Raul Chavez, Eastern Mennonite Uni- 
versity; Darrell Cousert, University of Indianapolis; 
David Dorsett, Florida Institute of Technology; 
Michael P. Dumler, Illinois State University; Kathy 
Dye, Thomas More College; Don English, Texas 
A&M University-Commerce; Antonnia Espiritu, 


Hawaii Pacific University; Hamid Falatoon, Univer- 
sity of Redlands; Judson Faurer, Metropolitan State 
College of Denver; Eve Fogarty, New Hampshire 
College; Bob Folden, Texas A&M University—Com- 
merce; Gary Grudintski, San Diego State Univer- 
sity; John Hanke, Eastern Washington University; 
Alan G. Heffner, Silver Lake College; Lee H. Igel, 
New York University; Burt Kaliski, New Hampshire 
College; Jane Legacy, Southern New Hampshire 
University; Andrew Luna, State University of West 
Georgia; Andrew Lynch, Southern New Hamp- 
shire University; Iraj Mahdvi, National University; 
Judith McKnew, Clemson University; Rosemarie 
Reynolds, Embry Riddle Aero University—Daytona; 
Randi L. Sims, Nova Southeastern University; Gary 
Stark, Northern Michigan University; Bruce Strom, 
University of Indianapolis; Cecelia Tempomi, 
Southwest Texas State University; Charles War- 
ren, Salem State College; Dennis G. Weis, Alliant 
International University; Bill Wresch, University of 
Wisconsin-Oshkosh; and Robert Wright, University 
of Illinois at Springfield. 


We are also indebted to dozens of students who identi- 
fied areas of confusion so that we could make concepts 
more understandable, who participated in search tests, 
who worked on numerous research projects demonstrat- 
ing where we needed to place more emphasis, and who 
reminded us with their questions and actions that many 
aspects of the research process operate below their learn- 
ing radar. 

Through this 12th edition, we hope you and your stu- 
dents discover, or rediscover, how stimulating, challeng- 
ing, fascinating, and sometimes frustrating this world of 
research-supported decision making can be. 


Pamela Schindler 
Donald Cooper 
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In its 12th edition, Business Research Methods, all chap- 
ters have been evaluated for currency and accuracy. Revi- 
sions were made to accommodate new information and 
trends in the industry. The book has a new structure and 
the Online Learning Center has been enhanced. 


¢ The book chapter structure was changed by moving 
Multivariate Analysis: An Overview to the Online 
Learning Center, and renumbering the last two 
chapters, making the book 20 chapters in all, and 
by moving several end-of-chapter appendices to the 
Online Learning Center, as noted in the for-each-of- 
the-chapters section below. 


e We’ve removed the feature we called a pulsepoint— 
the pullout statistic drawn from a research project— 
to streamline the appearance of the chapter, and 
because this element was not a reviewer favorite. 


e The Cases section contains the abstract for the new 
case: Marcus Thomas LLC Tests Hypothesis for 
Troy-Bilt Creative Development, and an updated 
case-by-chapter suggested use chart. 


e Appendix A includes the newly redesigned Mind- 
Writer CompleteCare online survey. 


e The 2012 Online Learning Center has new mate- 
rial including: a new case, Marcus Thomas LLC 
Tests Hypothesis for Troy-Bilt Creative Develop- 
ment, complete with instrument; the permissioned 
report Business Uses of Census Data and Nielsen 
Company Capabilities; EyeTrackShop’s example 
report Visual Effectiveness Research on McDonald's 
YouTube Ad; the following appendices: Appendix 
la: How the Research Industry Works (updated), 
Appendix 5a: Bibliographic Database Searches, Ap- 
pendix 5b: Advanced Database Searches (updated), 
Appendix 9a: Complex Experimental Designs, Ap- 
pendix 9b: Test Markets, Appendix 13b: Pretesting 
Options and Discoveries; and the chapter Multivari- 
ate Analysis: An Overview. 


e The Instructor’s Manual contains new research ex- 
amples for discussion or testing, as noted below. 


For Each of the Chapters. A detailed listing of 
chapter-by-chapter changes is provided here for your 
convenience. 


e Chapter 1 The following elements are new to this 
edition: the chapter-opening pull quote; opening 
paragraph expounding on the theme relating to cloud 
technology and research; three new Snapshots: on 
Mercedes-Benz Snapshot of the Stars Insight online 


research community, on the client perspective of 
consultancy skills needed by researchers, and on 
pattern thinking at YUM!; updates to the factors that 
encourage the studying of research methods; new 
NTT Communications ad with caption relating to the 
importance of information to business; a PicProfile 
on NextGen Marketing Research blog and website 
screenshot, related to blogs as an online training 
source for researchers; a new exhibit Where Business 
Collects Research Information and modifications to 
two Exhibits: the research process and what guar- 
antees good research; and a new from-the-headlines 
discussion question. Additionally, content and data 
updates have been made as needed to reflect the most 
current information available. The following ele- 
ment was moved to a different chapter: Snausages 
(PicProfile to Chapter 12). The following elements have 
been moved to the Instructor’s Manual (IM) as ad- 
ditional discussion opportunities: Mary Kay (Snap- 
shot) and MinuteMaid research model (Exhibit). The 
following items were removed: Maritz Research ad, 
Yahoo Banner ads (Snapshot). The following end-of- 
chapter appendix moves to the Online Learning Cen- 
ter: Appendix la: How the Research Industry Works. 


Chapter 2 The following elements are new to this 
edition: the chapter-opening pull quote; two new 
Snapshots: on privacy issues related to location- 
based services, and on ethics related to mobile 
surveys and ESOMAR and CASRO. Statistical 
updates have been made in the Snapshot Has Trust 
Trumped Privacy? New images have been added 
related to privacy issues related to children and to 
privacy related to hard-to-reach sample subjects. We 
added a new from-the-headlines discussion ques- 
tion and made changes and additions to Exhibit 2-5: 
Resources for Ethical Awareness. The following 
elements were moved to the IM: Snapshots related 
to TrustE (Engendering Trust Online) and tracking 
online behavior (Google: Tracking Search Patterns). 


Chapter 3 The following elements are new to this 
edition: the chapter-opening pull quote; a new 
PicProfile related to what influences online purchasers to 
abandon; the images related to the use of observation 
research and curiosity; and a new from-the-headlines 
discussion question. The following items were moved 
to the IM: the Snapshot related to researching emerg- 
ing issues (Forrester Research: Can an Auto Dealer- 
ship Go Lean?). The Synovate ad was removed. 


Chapter 4 The following elements are new to 
this edition: the chapter-opening pull quote; new 
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Snapshot related to embedding feedback mecha- 
nisms in websites on The Container Store and 
OpinionLab; new PicProfile relating to emerging 
techniques in research from the GRIT report; new 
Snapshot related to use of video and online research 
involving new and expecting moms; new Decipher 
ad relating methodology choice to avoiding the 
favored-technique syndrome; and a new from-the- 
headlines discussion question. Modifications were 
made to Exhibit 4-1: The Research Process to re- 
flect the new chapter structure, and changes to the 
text related to the ongoing study of CompleteCare. 
A making-research-decisions discussion question 
was eliminated and replace with an Apple-related 
question. The iThink Meet your Focus Group ad 
was removed. The following elements were moved 
to the IM: the Snapshots Should Companies Hire 
Teens? and Covering Kids: The Management- 
Research Question Hierarchy, and the PicProfile 
related to research for Kraft cheese. 


Chapter 5 The following elements are new to 

this edition: the chapter-opening pull quote; new 
screenshot of USA.gov website; a new Snapshot 

on Odin Text, a text analytics software for do-it- 
yourself researchers; a new from-the-headlines 
discussion question. A major change that is reflected 
in this and other chapters is the repurposing of the 
MindWriter CompleteCare research project as an 
online survey; this resulted in significant changes to 
the CloseUp: Proposing Research for MindWriter 
and the Exploration Phase. Additionally, modi- 
fications were made to the Snapshot Blogs: Gold 
Mine or Land Mine? to reflect current statistics and 
trends, and to the Snapshot Surfing the Deep Web 

to update sources, and to the Snapshots How Will 
Cloud Computing Affect Research?, Mining the Web 
for Feelings ?, and The Online Professional Commu- 
nity as a Source of Business Information to reflect 
current thinking and new examples. A making- 
research-decisions discussion question was elimi- 
nated and replace with a TJMaxx-related question. 
The following element was moved to the IM: the 
Snapshot $/ Million Data Analysis Prize Improves 
Netflix Movie Predictions. The following appendi- 
ces were moved to the Online Learning Center: Ap- 
pendix 5a: Bibliographic Database Searches (with 
updates to Exhibits 5a-2 and 5a-3) and Appendix 5b: 
Advanced Database Searches. 


Chapter 6 The following elements are new to this 
edition: the chapter-opening pull quote; text changes 
to reflect the repurposing of the CompleteCare 
research as an online survey and new statistics on 
qualitative research; a new Snapshot with image on 
the McDonald’s listening tour research; a new Luth 


Research image related to permission-based online 
tracking research; a new image and caption related 
to sample selection; and a new from-the-headlines 
discussion question on Kraft renaming of its inter- 
national snack food business. The following ele- 
ments were moved to the IM: the Snapshots Cheskin 
Knows Teens, Smith Barney’s Benchmark Job Envi- 
ronment Research, and The Ohio Lottery Initiates a 
Two-Stage Study, and the PicProfile on the United 
States Tennis Association; a discussion question on 
Tide Basics. The Zoomerang ad was removed. 


Chapter 7 The following elements are new to 
this edition: the chapter-opening pull quote; text 
changes to reflect new statistics on qualitative re- 
search and emerging or strengthening methodolo- 
gies; new source for Exhibit 7-2; a new PicProfile 
on Hallmark’s storybooks; a new Snapshot of 
mystery shopping at Office Depot; a new from- 
the-headlines discussion question on Pepsi’s “Live 
for Now” campaign. The following elements were 
modified: the Snapshot What Does Cyberspace 
Offer for Performance Review Research? was 
changed to reflect a change in company name and 
add a new example. The NCR discussion question 
was moved to the making-research-decisions sec- 
tion. The following elements were moved to the IM: 
the Snapshots Hamilton Beach: Right Blend(er) for 
Mexico, but Not for Europe and Hallmark: Quali- 
tative Research Enriches Sinceramente Hallmark. 
The following element was eliminated: a making- 
research-decisions discussion question related to the 
removed Hallmark Snapshot. 


Chapter 8 The following elements are new to this 
edition: the chapter-opening pull quote; a new Snap- 
shot related to neuroscience and PET scan in busi- 
ness research; a new Snapshot on Web-based eye 
tracking research; a new image included with the 
Snapshot When Physicians and Their Patients Are 
Research Subjects; and a new from-the-headlines 
discussion question on Walmart. Modifications 
were made to the following elements: the Snapshot 
Walmart Boosts RFID Technology for Observation; 
adjustment to discussion question 6. The following 
elements were moved to the IM: the Snapshots Peo- 
ple Meters Go Personal, New Mexico’s Longitudinal 
Observation Study of Seatbelt Use, and SizeUSA. 


Chapter 9 The following elements are new to 
this edition: the chapter-opening pull quote; new 
Snapshot on Wendy’s new burger test market; 
new Snapshot on a beverage message experiment; 
and a new from-the-headlines discussion question 
related to mergers. The following elements were 
changed: new graphic within the Snapshot Online 
Dating Industry Claims Vs. Kissing a Lot of Frogs 


to reflect new data. The following elements were 
moved to the Online Learning Center: Appendices 
9a: Complex Experimental Designs and 9b: Test 
Markets. The following elements were moved to the 
IM: PicProfile on Best Buy; and Snapshots 7:G./. 
Friday’s Experiments with Smaller Portions and A 
Nose for Problem Odors. 


Chapter 10 The following elements are new to 
this edition: the chapter-opening pull quote; a new 
Snapshot on gamification in research; PicProfile 
related to interview methodology use over time; 
in-text changes related to updating statistics and, 
within Exhibits, company name changes; a new 
PicProfile related to mixed-mode research; statis- 
tics updates within the Snapshot Are Cell Phones 
and Smartphones Ready for Research?; a new 
PicProfile on telephone research response rates; in- 
text changes related to the repurposed MindWriter 
CompleteCare project; a new from the headlines 
discussion question related to voice-of-consumer 
(VoC) research; a reference in the case listing for 
the new case: Marcus Thomas LLC Tests Hypothesis 
for Troy-Bilt Creative Development. The follow- 
ing elements were moved to the IM: the Snapshots 
Starwood Hotels Measures the Power of Comments, 
Radio and Research, and Aleve: Personal Interviews 
Provide Relief. The following elements were elimi- 
nated: PicProfile on RTI call center. 


Chapter 11 The following elements are new to this 
edition: the chapter-opening pull quote; new image 
related to the in-text example; new image and new 
in-text changes about nominal scales related to that 
image; a new Snapshot related to talent analytics in 
HR; replacement of discussion question 8 and a new 
from-the-headlines discussion question. The fol- 
lowing element was moved to the IM: the Snapshot 
Measuring Attitudes about Copyright Infringement. 
The image of LeBron James was removed. 


Chapter 12 The following elements are new to this 
edition: the chapter-opening pull quote; the iThink 
image related to online focus groups; the Snausages 
PicProfile; a new Snapshot related to Maritz and 
how it measures customer satisfaction; a new 
PicProfile related to branching questions executed 
with SurveyMonkey online software; new image re- 
lated to shopping research; changes to the CloseUp 
related to the MindWriter CompleteCare project due 
to repurposing the study as an online survey; and a 
new from-the-headlines discussion question related 
to the iPad. The following element was moved to 
the IM: the Snapshot Measuring Respect. 


Chapter 13 The following elements are new to this 
edition: the chapter-opening pull quote; a new Verint 
dashboard image related to managing information 


>detailed changes to this edition xvii 


from different sources; a new Exhibit 13-3 on fac- 
tors affecting respondent honesty; a new Snapshot 
relating to challenges and Solutions to Mobile 
Questionnaires; updates to Exhibit Sources of Ques- 
tions; updates to Sample Components of Commu- 
nication Instruments to reflect online surveys; and 
major change to the CloseUp: Instrument Design 
for MindWriter to include the new online survey for 
the CompleteCare research project; a new from-the- 
headlines discussion question related to the employ- 
ment; and a reference in the case listing for the new 
case: Marcus Thomas LLC Tests Hypothesis for 
Troy-Bilt Creative Development. The following ele- 
ment was moved to the IM: the Snapshot A Survey 
Cold as Ice. The following element was moved to 
the Online Learning Center: Appendix 13b: Pretest- 
ing Options and Discoveries. 


Chapter 14 The following elements are new to this 
edition: the chapter-opening pull quote; a new Serta 
image; a new Snapshot on using charitable dona- 
tions to incentivize participation; a new Snapshot on 
mixed-access sampling; in-text changes to Chinese 
cell phone use statistics; a new from-the-headlines 
discussion question about Nike product introduc- 
tions; and a reference in the case listing for the 

new case: Marcus Thomas LLC Tests Hypothesis 
for Troy-Bilt Creative Development. The follow- 

ing elements were moved to the IM: the Snapshots 
Creating Samples: Then and Now and New Product 
Research Blind Spot. 


Chapter 15 The following elements are new to 
this edition: the chapter-opening pull quote; new 
image related to data quality; a new image for 
XSight software; a new caption related to the IBM® 
SPSS® image; a new from-the-headlines discussion 
question related to data sourcing solutions. The fol- 
lowing element was modified: a statistical change to 
the Snapshot CBS: Some Labs Are Extraordinary. 


Chapter 16 The following elements are new to 
this edition: the chapter-opening pull quote; a new 
Snapshot on Novations data visualization; Exhibits 
16-2 and 16-3 are new and relate to social media 
use and text references to these Exhibits have been 
changed as well; all the tables within the CloseUp 
Using Tables to Understand Data are new; a new 
geospatial image from RealtyTrac; a new Snapshot 
on digital natives; and a new from-the-headlines 
discussion question related to tablet apps. The 
following modifications were made: the Snap- 
shot Internet-Age Researchers: Building Critical 
Transferable Skills was moved within the chapter; 
art modifications to the graph within the Snapshot 
Empowering Excel; the Snapshot Internet-Age 
Researchers: Building Critical Transferable Skills 


xviii 


>detailed changes to this edition 


has moved to later in the chapter. The following 
elements were moved to the IM: the Snapshot 
Extensive Research Launches Starbucks Card 
Duetto™ Visa. The following element was elimi- 
nated: The DDW image. 


Chapter 17 The following elements are new to 
this edition: the chapter-opening pull quote; a new 
Prius image and caption with updated information; 
a new Snapshot about hypothesis testing; new Snap- 
shot on A/B testing; and a new from-the-headlines 
discussion question on aptitude tests. The follow- 
ing elements were moved to the IM: the Snapshot 
Direct-to-Consumer Ads under Heavy Fire; the 
Snapshot Research beyond the Clip. The following 
elements were modified: the Snapshot Testing a Hy- 
pothesis of Unrealistic Drug Use in Movies. 


Chapter 18 The following elements are new to 
this edition: the chapter-opening pull quote; a new 
Snapshot about the Oscars; a new image with the 
Constellation Wines PicProfile; and a new from-the- 
headlines discussion question related to correlation 
and the building of skyscrapers. This following ele- 
ments were moved to the IM: the Snapshot What’s 
a Business Education without Wine? The following 
elements were modified: the Snapshot Envirosell: 
Studies Reveal Left-Hand Retail was moved later in 
the chapter; updates were made to the Snapshot Ad- 
vanced Statistics Increase Satisfaction and Release 
More Funds through ATMs. 


Chapter 19 This was previously Chapter 20. The 
following elements are new to this edition: the 
chapter-opening pull quote; a new image related to 
proofing; changes to the CloseUp MindWriter Writ- 
ten Report to reflect the redesigned project as an on- 
line survey; new in-text content under Semi-Tabular 
and Tabular headings; new content in Exhibit 19-5 
Sample Tabular Finding; new data and graphs for 
Exhibit 19-8: U.S. Truck Sales; and a new from- 
the-headlines discussion question related to small 
business optimism. The following elements were 
modified: Exhibit 19-7: Cable Subscribers. 


Chapter 20 This was previously Chapter 21, which 
was totally new in the last edition. The following el- 
ements are new to this edition: the chapter-opening 
pull quote; new image related to online presenta- 
tions; new information in Exhibit 20-13: Using Key 
Word Prompts as a Substitute for Notes; new in-text 
table related to business jargon; a new PicProfile re- 
lated to the use of pictographs in oral presentations; 
and a new from-the-headlines discussion question 
related to a famous trial. The following elements 
were modified: 10 Exhibits (20-2, 20-3, 20-4, 20-5, 
20-6, 20-7, 20-8, 20-10, 20-11, and 20-15) to en- 
hance understanding through a different use of color 
or design. 


Preface xéii >pa rt IV 


>part | Analysis and Presentation 


Introduction to Business of Data 373 


Research 15 Data Preparation and Description 374 
Appendix 15a: Describing Data Statistically 398 


—s 


Research in Business 2 
16 Exploring, Displaying, and Examining Data 404 


2 Ethics in Business Research 26 
Le 17 Hypothesis Testing 428 
3. Thinking Like a Researcher 48 
; 18 Measures of Association 466 
4 ~The Research Process: An Overview 74 
Siacniaaihe Ec Gussaeaat h 19 Presenting Insights and Findings: 
5 arifying the Research Ques ip roug Written Reports 502 
Secondary Data and Exploration 92 
20 Presenting Insights and Findings: 
Oral Presentations 538 
>part Il 
The Design of Business >case index = 575 
Research 121 
Research Design: An Overview 122 >appendices 
Qualitative Research 142 A Business Research Requests and Proposals 


(with Sample RFP) 586 


6 
7 
8 Observation Studies 170 
9 Focus Group Discussion Guide 610 


Experiments 190 
Nonparametric Significance Tests 612 


D Selected Statistical Tables 619 


10 Surveys 214 


>part ll References 630 
The Sources and Collection Glossary 652 
of Data 243 Photo Credits 670 


Indexes 671 
11. Measurement 244 


12 Measurement Scales 266 
13 Questionnaires and Instruments 294 


Appendix 13a: Crafting Effective Measurement 
Questions 328 


14 Sampling 336 
Appendix 14a: Determining Sample Size 364 


xix 


Preface xii 


>part | 


1 


Introduction to Business 
Research ; 


Research in Business 2 
Why Study Business Research? 4 


Information and Competitive Advantage 7 
Goals 7 
Decision Support 8 
Business Intelligence 9 
Strategy 9 
Tactics 10 


Hierarchy of Information-Based Decision 
Makers 11 


The Research Process: A Preview 12 


Is Research Always Problem- 
Solving Based? 15 


What Is Good Research? 15 
A Glimpse at Four Research Studies 18 
ClassicToys 18 
MedImage 19 
MoreCoatings 19 
York College 19 
What Dilemma Does the Manager Face? 


The Types of Research Studies Represented 
by the Four Examples 21 


>summary 23 

>key terms 23 
>discussion questions 24 
>cases 24 


Ethics in Business Research 26 


What Are Research Ethics? 28 
Ethical Treatment of Participants 28 
Benefits 29 
Deception 30 
Informed Consent 31 


Debriefing Participants 32 
Rights to Privacy 32 
Data Collection in Cyberspace 35 
Ethics and the Sponsor 36 
Confidentiality 36 
The Sponsor—Researcher Relationship 37 
Sponsor’s Ethics 39 
Researchers and Team Members 39 
Safety 39 
Ethical Behavior of Assistants 40 
Protection of Anonymity 40 
Professional Standards 40 
Resources for Ethical Awareness 42 
>summary 45 
>key terms 45 
>discussion questions 46 
>cases 47 


Thinking Like a Researcher 48 


The Language of Research 50 
Concepts 50 
Constructs 52 
Definitions 52 
Variables 55 
Propositions and Hypotheses 58 
Theory 61 
Models 63 
Research and the Scientific Method 64 
Sound Reasoning for Useful Answers 66 
>summary 70 
>key terms 71 
>discussion questions /71 
>cases 73 


The Research Process: 
An Overview 74 

The Research Process 76 
Stage 1: Clarifying the Research Question 77 
Stage 2: Proposing Research 80 


Resource Allocation and 
Budgets 80 


Valuing Research Information 81 
The Research Proposal 82 


Stage 3: Designing the Research 
Project 82 


Research Design 82 
Sampling Design 84 
Pilot Testing 85 


Stage 4: Data Collection and 
Preparation 85 


Stage 5: Data Analysis and 
Interpretation 86 


Stage 6: Reporting the Results 86 

Research Process Issues 87 
The Favored-Technique Syndrome 87 
Company Database Strip-Mining 88 
Unresearchable Questions 88 
Ill-Defined Management Problems 88 
Politically Motivated Research 89 

>summary 89 

>key terms 90 

>discussion questions 90 

>cases 91 


Clarifying the Research Question 


through Secondary Data and 
Exploration 92 


A Search Strategy for Exploration 94 
Levels of Information 96 
Types of Information Sources 97 
Evaluating Information Sources 100 
Mining Internal Sources 102 
Evolution of Data Mining 103 
Data-Mining Process 105 


The Question Hierarchy: How Ambiguous 
Questions Become Actionable 
Research 108 


The Management Question 108 
The Research Question 112 
Investigative Questions 113 
Measurement Questions 118 
>summary 118 
>key terms 119 
>discussion questions 119 
>cases 120 


>part Il 


The Design of Business 


Research 121 


>contents 


6 Research Design: An Overview 122 
What Is Research Design? 


Classification of Designs 


Exploratory Studies 


129 


Qualitative Techniques 


Secondary Data Analysis 


Experience Survey 


Focus Groups 


Two-Stage Design 


Descriptive Studies 


Causal Studies 


>summary 139 


>key terms 


136 


140 


131 


133 


133 
134 


>discussion questions 


>cases 


vf Qualitative Research 


141 


124 


129 


140 


What Is Qualitative Research? 


Qualitative versus Quantitative 
144 


The Controversy 


Research 


The Distinction 


144 


146 


The Process of Qualitative 
148 


Qualitative Research Methodologies 


Research 


Sampling 


Interviews 


151 
152 


Individual Depth Interviews 


Group Interviews 


157 


Combining Qualitative 


Methodologies 


Case Study 


165 
165 


Action Research 


166 


126 


130 


142 
144 


156 


Merging Qualitative and Quantitative 


Methodologies 


>summary 167 


>key terms 


166 


168 


>discussion questions 


>cases 


169 


168 


151 


xxii >contents 


8 Observation Studies 170 


The Uses of Observation 172 
Nonbehavioral Observation 173 
Behavioral Observation 175 

Evaluation of the Observation Method 176 

The Observer—Participant Relationship 176 
Directness of Observation 177 
Concealment 177 
Participation 178 

Conducting an Observation Study 178 
The Type of Study 178 
Content Specification 180 
Observer Training 181 
Data Collection 181 

Unobtrusive Measures 184 

>summary 18/7 

>keyterms 188 

>discussion questions 188 

>cases 189 


9 Experiments 190 


What Is Experimentation? 192 

An Evaluation of Experiments 193 
Advantages 193 
Disadvantages 193 

Conducting an Experiment 194 
Selecting Relevant Variables 195 
Specifying Treatment Levels 195 


Controlling the Experimental 
Environment 197 


Choosing the Experimental Design 197 
Selecting and Assigning Participants 199 
Pilot Testing, Revising, and Testing 199 
Analyzing the Data 199 

Validity in Experimentation 201 
Internal Validity 201 
External Validity 203 

Experimental Research Designs 204 
Preexperimental Designs 204 
True Experimental Designs 206 


Field Experiments: Quasi- or 
Semi-Experiments 207 


>summary 210 
>key terms 211 


>discussion questions 211 
>cases 212 


1 0) Surveys 214 
Characteristics of the Communication 
Approach 216 
Error in Communication Research 219 


Choosing a Communication 
Method 224 


Self-Administered Surveys 226 


Evaluation of the Self-Administered 
Survey 226 


Maximizing Participation in the 


Self-Administered Survey 230 
Self-Administered Survey Trends 230 
Survey via Telephone Interview 232 
Evaluation of the Telephone Interview 232 
Telephone Survey Trends 236 
Survey via Personal Interview 237 


Evaluation of the Personal Interview 


Survey 238 
Selecting an Optimal Survey Method 238 
Outsourcing Survey Services 289 
>summary 240 
>key terms 241 
>discussion questions 241 
>eases 242 


>part Ill 


The Sources and Collection 
of Data 243 


1 1 Measurement 244 


The Nature of Measurement 246 
What Is Measured? 248 
Measurement Scales 249 
Nominal Scales 250 
Ordinal Scales 252 
Interval Scales 253 
Ratio Scales 253 
Sources of Measurement Differences 254 
Error Sources 256 
The Characteristics of Good Measurement 257 
Validity 257 


Content Validity 257 
Criterion-Related Validity 258 
Construct Validity 259 
Reliability 260 

Practicality 262 


>contents 


Type of Scale for Desired Analysis 297 
Communication Approach 298 
Disguising Objectives and 

Sponsors 299 

Preliminary Analysis Plan 300 


xxiii 


Phase 2: Constructing and Refining the 
Measurement Questions 301 


>summary 263 


>key terms 263 
Question Categories and Structure 302 


Question Content 302 
Question Wording 304 
Response Strategy 306 


>discussion questions 263 
>cases 265 


1 2 Measurement Scales 266 


The Nature of Attitudes 268 


The Relationship between Attitudes 
and Behavior 270 


Attitude Scaling 270 


Selecting a Measurement Scale 271 


Sources of Existing Questions 312 


Phase 3: Drafting and Refining the 
Instrument 315 


Participant Screening and Introduction 316 
Measurement Question Sequencing 317 
Instructions 320 
Conclusion 3283 
Overcoming Instrument Problems 323 
The Value of Pretesting 324 

>summary 324 


Research Objectives 271 
Response Types 271 
Data Properties 272 
Number of Dimensions 272 
Balanced or Unbalanced 272 
Forced or Unforced Choices 272 
Number of Scale Points 273 
Rater Errors 274 

Rating Scales 275 
Simple Attitude Scales 275 
Likert Scales 278 
Semantic Differential Scales 280 


Numerical/Multiple Rating List 
Scales 283 


Stapel Scales 284 
Constant-Sum Scales 284 
Graphic Rating Scales 284 


>key terms 325 
>discussion questions 325 
>cases 32/ 


Appendix 13a: Crafting Effective 
Measurement Questions 328 


14 Sampling 336 


The Nature of Sampling 338 
Why Sample? 3388 
What Is a Good Sample? 389 
Types of Sample Design 341 
Steps in Sampling Design 344 
Ranking Scales 285 What Is the Target Population? 345 
Sorting 287 What Are the Parameters of 


Cumulative Scales 289 Interest? 345 
What Is the Sampling Frame? 347 


What Is the Appropriate Sampling Method? 348 
What Size Sample Is Needed? 348 
Probability Sampling 349 
Simple Random Sampling 349 
Complex Probability Sampling 350 


>summary 290 

>key terms 291 
>discussion questions 292 
>cases 293 


1 3 Questionnaires and 


Instruments 294 Nonprobability Sampling 358 


Phase 1: Revisiting the Research Question Practical Considerations 358 
Hierarchy 296 Methods 359 


Xxiv >contents 


>summary 361 Pareto Diagrams 415 

>key terms 362 Boxplots 415 

>discussion questions 362 Mapping 418 

>cases 363 Cross-Tabulation 419 

Appendix 14a: Determining Sample The Use of Percentages 420 
Size 364 Other Table-Based Analysis 423 


>summary 424 
>key terms 425 


>pa rt IV >discussion questions 425 
Analysis and Presentation pcases 426 
of Data 3:73 
1 7 Hypothesis Testing 428 
15 bata Preparation and lnttoduction 430 

Description 374 Statistical Significance 430 

Introduction 376 The Logic of Hypothesis Testing 482 

Editing 377 Tests of Significance 440 
Field Editing 377 Types of Tests 440 
Central Editing 377 How to Selecta Test 442 

Coding 379 Selecting Tests Using the Choice Criteria 448 
Codebook Construction 380 One-Sample Tests 444 
Coding Closed Questions 380 Two-Independent-Samples Tests 447 
Coding Open-Ended Questions 382 Two-Related-Samples Tests 450 
Coding Rules 383 k-Independent-Samples Tests 458 
Using Content Analysis for Open k-Related-Samples Tests 460 
Questions 384 >summary 463 
“Don’t Know” Responses 387 >key terms 463 
Missing Data 389 >discussion questions 464 

Data Entry 391 >cases 465 


Alternative Data Entry Formats 391 
On the Horizon 395 


18 Measures of Association 466 
>summary 395 


>key terms 396 Introduction 468 
>discussion questions 396 Bivariate Correlation Analysis 469 
>cases 397 Pearson’s Product Moment Coefficientr 469 


Appendix 15a: Describing Data Scatterplots for Exploring Relationships 470 


Statistically 398 The Assumptions oft 473 
Computation and Testing ofr 473 
1 6 Exploring, Displaying, Simple Linear Regression 479 
and Examining Data 404 The Basic Model 479 
Exploratory Data Analysis 406 Concept Application 480 


Frequency Tables, Bar Charts, and Pie Method of Least Squares 482 
Charts 407 Predictions 486 


Histograms 408 Testing the Goodness of Fit 487 
Stem-and-Leaf Displays 411 Nonparametric Measures of Association 490 


>contents XXV 


Measures for Nominal Data 490 Audience Analysis 544 
Measures for Ordinal Data 494 Types of Learners 545 
>summary 498 Keep Your Audience from Checking Out 546 
>key terms 499 Planning and the Web-Delivered 
>discussion questions 499 Presentation S47 
seases 501 Organize 547 
Traditional Patterns of Organization 548 
1 9 Presenting Insights and Findings: The Motivated Sequence Organization 549 
Written Reports 502 The Narrative Organization 549 


The Rule of Three and the Three-Point 
Speech 551 


Support 551 


Introduction 504 
The Written Research Report 504 


Short Reports 504 Conveying Personal Experience through 
Long Reports 505 Stories 553 


Research Report Components 507 Demonstrations 554 


Prefatory Items 508 Visualize 554 


Introduction S09 Psychological and Physical Foundations 555 


Methodology 510 Design Principles 557 
Findings 511 Prescriptions for Better Slides 561 
Conclusions 512 Deliver 562 
Appendices 512 Modes of Delivery 562 
Bibliography 512 Scripts and Notes 562 
Writing the Report 512 Details Make a Difference 563 
Prewriting Concerns 512 Practice and Arrange 566 
Writing the Draft 514 Rehearsal Is Essential 566 
Presentation Considerations 517 Controlling Performance Anxiety 567 
Presentation of Statistics 517 Arrangements for Facilities and 
Text Presentation 517 Equipment S69 
Semitabular Presentation 529 >summary 5/71 
Tabular Presentation 529 >key terms 5/3 
Graphics 531 >discussion questions 573 
>summary 536 >cases 5/74 


>key terms 536 
>discussion questions 537 


>case index — sz 


>cases 537 


20 Presenting Insights and Findings: >appendices 585 


Oral Presentations 538 
a Business Research Requests 
and Proposals (with Sample 


Introduction 540 


Aristotle’s Three Principles of Persuasive 


Communication 541 RFP) 586 
Ethos 542 Proposing Research 586 
Pathos 542 The Request for Proposal (RFP) 586 
Logos 542 Creating the RFP 586 


Plan 543 The Research Proposal 590 


XXVi >contents 


Sponsor Uses 592 b Focus Group Discussion Guide 610 
Researcher Benefits 593 Background 610 
Types of Research Proposals 593 


Internal Proposals 595 c Nonparametric Significance 


External Proposals 595 Tests 612 
Structuring the Research Proposal 595 One-Sample Test 612 
Executive Summary 595 Kolmogorov-Smirnov Test 612 
Problem Statement 596 Two-Samples Tests 613 
Research Objectives 596 Sign Test 613 
Literature Review 596 Wilcoxon Matched-Pairs Test 613 
Importance/Benefits of the Study S97 Kolmogorov-Smirnov Two-Samples Test 614 
Research Design 597 Mann-Whitney U Test 615 
Data Analysis 597 Other Nonparametric Tests 617 
Nature and Form of Results 597 k-Samples Tests 617 
Qualifications of Researchers 598 Kruskal-Wallis Test 617 
Budgel 228 d_ Selected Statistical Tables 619 
Schedule 599 
Facilities and Special Resources 600 
Project Management 600 References 630 
Bibliography 601 Glossary 652 
aypeneces: 204 Photo Credits 670 
Evaluating the Research Proposal 601 
Covering Kids RFP 606 Indexes 
Background 606 Name 671 
Contractual Needs 607 Company 675 
Evaluation Task 607 Subject 679 


Anticipated Time Schedule 608 
Proposal Instructions 608 
Review Criteria 609 


Chapters 2-5 


Exploration Exploration 


_ Research \ Appendix A 
Proposal | 


Research Design 
Strategy 


(type, purpose, time frame, scope, environment) Chapters 6-14 
Data Collection Sampling 
Design Design 
Data Collection & Preparation Chapter 15 


Chapters 16-18 


Research Reporting 
Chapters 19-20 


Management 
Decision 


Introduction to Business Research 


chapter 1 


learningobjectives 
After reading this chapter, you should understand .. . 


What business research is and how it differs from decision support systems and business intelligence systems. 
The trends affecting business research and the emerging hierarchy of research-based decision makers. 

The different types of research studies used in business. 

The distinction between good business research and that which falls short of professional quality. 


The nature of the research process. 


6 6 Forward-thinking executives recognize that 
analytics may be the only true source of 
sustainable advantage since it empowers 
employees at all levels of an organization 


with information to help them make smarter 
decisions. 9 9 
Wayne Eckerson, 


director of research, business applications and architecture group, 
TechTarget 


bringingresearchtoolife 6° 


Newly promoted to her position, Wines has a 
TV journalism and government public relations 
background. She has been a MindWriter laptop owner 
since it came on the market decades earlier and has 
never personally experienced a problem. She wants a 
research supplier from whom she can learn, as well as 
one whom she can trust to do appropriate, high-quality 
research. 

The last interviewee is Jason Henry, managing 
partners, Henry & Associates. H&A comes highly 
recommended by a professional colleague in a different 
industry. H&A has gained a reputation for merging 
traditional methodologies with some creative new 
approaches. Myra is interested in exploring the firm’s 
methodology for customer satisfaction studies. As Wines 
approaches Henry in the waiting area, she extends her 
hand. “Welcome to MindWriter, Jason. ’m Myra Wines.” 

Henry rises, clasping Wines’s hand in a firm hand 
shake. “Pleased to meet you, Myra.” 

Myra directs Jason’s attention to a long corridor. 
“My team members are gathered in our conference 
room just down this hall. Let’s join them, shall we?” 

The interview process starts with Henry’s short pre- 
sentation on H&A and its capabilities. As the interview 
progresses, Henry shares some impressive results 
accomplished for former clients in noncompetitive 
industries. The last slide in his presentation features a 
top industry award H&A recently won for its customer 
satisfaction methodology. 

During the Q&A that follows, Henry demonstrates 
current knowledge of the computer industry (he’s 
obviously read numerous articles), confidence, and 
expertise, at a level that Wines initially had not expected 
given his relatively youthful appearance. At the conclusion 
of the interview, Wines is leaning toward hiring Henry & 
Associates, but wants to confer with her team. 


The next day, Myra calls Jason at his office. “We’ve 
chosen Henry & Associates for the MindWriter 
CompleteCare assessment contract. Congratulations.” 

“Thank you,” accepts Jason. “You’ve made the 
right choice.” 

“[ve got two seats on a flight to Austin next 
Wednesday,” shares Myra. “Can you join me? This 
will be my first look at the CompleteCare facility and 
my first face-to-face contact with its manager. I'd like 
someone along who can lay the groundwork for the 
project and understand the number crunching that’s 
already been done.” 

The phone goes silent as Jason pauses to consult 
his BlackBerry. Two internal meetings will need to 
be shifted, but MindWriter is an important new client. 
“Yes, Ican work that in as long as we’re back by 7 p.m. 
Ive got an evening commitment.” 

“Shouldn’t be a problem,” shares Myra. “Those 
seats I mentioned are on the corporate jet. We’ll be 
back by 5:30. I'll meet you in the lobby at the county 
airstrip at 8 a.m. Wednesday then.” 

“A quick question,” interrupts Jason before Myra 
can disconnect. “I need some idea of what’s happening 
at this meeting.” 

“The meeting is to get you started. I'll introduce 
you to other people you will be working with and 
share more details about the concerns we have with the 
CompleteCare program,” shares Myra. 

“Fine. Can you arrange a third seat? It would be 
best to include Sara Arens from the very beginning. 
Her expertise will be crucial to the success of the 
assessment program.” 

“Yes, you mentioned her before. That shouldn’t be a 
problem, but I'll check and get back to you.” 

“Then, Wednesday, Sara and I will plan on asking 
probing questions and listening to discover exactly what 


> Why Study Business Research? 


One of the fundamental shifts in organizational management in the last 10 years has been the purpose- 
ful integration of the Internet by managers at every level of decision making. It might be as simple 
as tracking project management through SharePoint sites or email messages, or as complex as call 
routing to various worldwide service centers to ensure the shortest wait time or using GPS to route 
delivery trucks to eliminate left turns. This integration wasn’t purposeful at the beginning of the last 
decade. Some have said that managers, many trained in lower-technology approaches, were actually 
dragged all but kicking and screaming into using the Internet as a workspace. But the speed of tech- 
nology change, and newer generations who have been raised with smart phones and tablet computers, 
have forced organizations to be more purposeful in their technology integration. Researchers, as part 
of those organizations, have been pulled along—some reluctantly, but increasingly with enthusiasm. 

By the Internet, we mean not just the company-generated content of the World Wide Web where 
information (and non-information) expands at an alarming rate, but also the rapid advance of user- 
generated content in social media. As data sources expand managers experience increasing pressure 
to make some sense of the massive amount of data generated. The Internet has also influenced the 
way in which we collaborate to define problems and opportunities, as well as our processes for in- 
formation collection and analysis brought about by the technologies that make up the space known 
as the “cloud.” It is in this space that research is experiencing fundamentally change, not just the 
tools of researchers but also the methodology of research. Thus we chose our cloud cover design to 
represent the theme of this edition: researchers are turning to new spaces to understand what moti- 
vates people and organizations, and understand processes and machines. They are using new tools to 
search for new business models, and to understand fundamental shifts in human behaviors, emotions, 
and attitudes. 

You are about to begin your study of business research, both the process and the tools needed to 
reduce risk in managerial decision making. Business research, as we use it in this text, is a system- 
atic inquiry that provides information to guide managerial decisions. More specifically, it is a process 
of planning, acquiring, analyzing, and disseminating relevant data, information, and insights to deci- 
sion makers in ways that mobilize the organization to take appropriate actions that, in turn, maximize 
performance. A variety of different types of research projects are grouped under the label “business 
research,” and we will explore them later in this chapter. 

Assume for the moment that you are the manager of your favorite full-service restaurant. You are 
experiencing significant turnover in your waiter/waitress pool, and some long-time customers have 
commented that the friendly atmosphere, which has historically drawn them to your door, is chang- 
ing. Where will you begin to try to solve this problem? Is this a problem for which research should 
be used? 

Perhaps you are the head of your state’s department of transportation, charged with determining which 
roads and bridges will be resurfaced or replaced in the next fiscal year. Usually you would look at the 
roads and bridges with the most traffic in combination with those representing the most economic disaster, 
if closed. However, the state’s manager of public information has expressed concern about the potential 
for public outcry if work is once again directed to more affluent regions of the state. The manager suggests 
using research to assist in making your decision, because the decision is one with numerous operational, 
financial, and public relations ramifications. Should you authorize the recommended research? 


>chapter 1 Research in Business 


Mercedes-Benz and TNS Infratest Develop Stars Insight 


Stars Insight is an online research collaboration between 
Mercedes-Benz (MB) and TNS Infratest (TNS). “The original 
purpose of Stars Insight is capturing insights on needs, val- 
ues, and attitudes of the important 20- to 45-year-old compact 
car driver,” shared Christian Bauer, MP/MR customer insights 
for Daimler AG. The insights would be used to craft marketing 
initiatives. 

TNS drew upon the strengths of Web 2.0 to deliver sev- 
eral key research assets: neutrality of moderation, researcher 
objectivity, data quality, and respondent anonymity while still 
allowing MB to narrow its focus to a particular target group of 
compact car owners. “Through a series of methods—includ- 
ing data from the recruiting screening process, statements 
from member’s profile pages, surveys (both preplanned and 
instantaneous), online discussions, participant blog posts, and 
hotspot creative competitions between community members— 
TNS is able to provide data that helps MB in several ways. Data 
create a picture of what a modern brand should be, identify 
future distribution channels, craft authentic target group com- 
munication measures by testing advertising, develop the ideal 
configuration of the desired automobile, identify target group 
purchase intention criteria, identify the price expectations in the 
compact-car segment, and understand modern mobility con- 
cepts,” explained Sandra Klaunzler, senior consultant automo- 
tive, TNS Infratest. 
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Interactivity is the hallmark of Stars Insight. For example, a “hot- 
spot is a period of 2-3 weeks where the community members deal 
with one specific topic—for example, what are appropriate ways 
to target young customers who might not yet drive or want to buy 
a Mercedes,” shared Kai Blask, associate director automotive for 
TNS Infratest. “Within the hotspot we assign the community mem- 
bers one specific task. For example, develop a communication 
strategy for the market launch of model XY. The respondents work 
on this either alone or in groups. All participants upload their con- 
cepts to the community on a specific date.” Member uploads are 
usually high-quality Word, PowerPoint, or other digital files. “After- 
wards, all other community members can evaluate the ideas and 
give 1 to 5 points for each idea, explained Blask. “The member’s 
idea with the best evaluations wins a specific prize or extra points,” 
as well as significant attention from Mercedes-Benz. 

Overall MB has 1,700 members in Stars Insight. Sixty per- 
cent of the members own a compact car (not a Mercedes- 
Benz) and 40 percent own a Mercedes-Benz (no matter what 
model/segment). TNS won the 2011 Best Study award from 
the German Association for Marketing and Social Research for 
the development of the Mercedes-Benz process. It leveraged 
Acquia Commons social business software to build the award- 
winning Mercedes-Benz social community website. 
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As the opening vignette and the early decision scenarios reveal, decision makers can be found 
in every type of organization: businesses, not-for-profit organizations, and public agencies. Re- 
gardless of where these decision makers are found or whether their resources are abundant or 
limited, they all rely on information to make more efficient and effective use of their budgets. 
Thus, in this book, we will take the broadest perspective of managing and its resulting application 
to business research. 

At no other time in our history has so much attention been placed on measuring and enhancing 
return on investment (ROD). At its most simplistic, when we measure ROI, we calculate the financial 
return for all expenditures. Increasingly, organizational managers want to know what strategies and 
tactics capture the highest return. In the last dozen years, as technology has improved our measure- 
ment and tracking capabilities, managers have realized they need a better understanding of employees, 
stockholders, constituents, and customer behavior in order to influence the desired metrics. Business 
research plays an important role in this new measurement environment. Not only does it help managers 
choose better strategies and tactics, but business research expenditures are increasingly scrutinized for 
their contribution to ROI. 

The research methods course recognizes that students preparing to manage any function—regardless 
of the setting—need training in a disciplined process for conducting an inquiry of a management 
dilemma, the problem or opportunity that requires a management decision. Several factors should 
stimulate your interest in studying research methods:!' 


1. Information overload. Although the Internet and its search engines present extensive amounts 
of information, the information’s quality and credibility must be continuously evaluated. The 
ubiquitous access to information has brought about the development of knowledge communi- 
ties and the need for organizations to leverage this knowledge universe for innovation—or risk 
merely drowning in data. 


2. Technological connectivity. Individuals, public-sector organizations, and businesses are adapt- 
ing to changes in work patterns (real-time and global), changes in the formation of relation- 
ships and communities, and the realization that geography is no longer a primary constraint. 
With the increased acceptance and use of mobile technology, information snacking, short 
online visits to get specific answers, has become the norm for information gatherers. This 
could have a profound influence on information collection designed to serve the needs of 
managers who want quick, smaller chunks of information, each of which is more decision rel- 
evant. While this influence is expected in quantitative techniques such as surveys, qualitative 
research is also increasingly embracing smaller iterative engagements with research subjects 
to drive research. 


3. Shifting global centers of economic activity and competition. The rising economic power of 
Asia and demographic shifts within regions highlight the need for organizations to expand their 
knowledge of consumers, suppliers, talent pools, business models, and infrastructures with 
which they are less familiar. This shift increases the value of research designs that can accom- 
modate different norms, values, technologies, and languages. Some in the research industry 
believe innovation in research methodology will come from the developing world, not the de- 
veloped economies, as countries in the developing world are already embracing mobile/social 
research methodologies to a greater degree. 


4. Increasingly critical scrutiny of big business. The availability of information has made it possible for 
all a firm’s stakeholders to demand inclusion in company decision making, while at the same time 
elevating the level of societal suspicion. Interconnected global systems of suppliers, producers, and 
customers have made the emergence and viability of megabusinesses not only possible, but likely. 


5. More government intervention. As public-sector activities increase, in order to provide 
some minimal or enhanced level of social services, governments are becoming increasingly 
aggressive in protecting their various constituencies by posing restrictions on the use of mana- 
gerial and business research tools (e.g., Do-Not-Call List, Spyware Act). 


6. Battle for analytical talent. Managers face progressively complex decisions, applying mathemati- 
cal models to extract meaningful knowledge from volumes of data and using highly sophisticated 
software to run their organizations. The shift to knowledge-intensive industries puts greater demand 
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on a scarcity of well-trained talent with advanced analytical skills. The integration of global labor 
markets, with its infusion of new talent sources, is only a partial answer. Many believe the value 

of research may no longer be in collecting data, but rather in focusing on context, implications, 
and outcomes. Data collection and even some major elements of analysis may become the future 
domain of technologists and data scientists. Researchers will be forced to offer new value based on 
strategic consulting principles, as data collection becomes more of a commodity. 


7. Greater computing power and speed. 

¢ Lower-cost data collection. Computers and telecommunications lowered the costs of data 
collection, drastically changing knowledge about consumers both at store and household 
levels; employees at the position, team, and department levels; suppliers and distributors 
at the transaction, division, and company levels; and equipment at the part, process, and 
production-run levels. 

¢ Better visualization tools. High-speed downloads of images allow us to help people visualize 
complex concepts; this enriches measurement capabilities. 

¢ Powerful computations. Sophisticated techniques of quantitative analysis are emerging to 
take advantage of increasingly powerful computing capabilities. 

¢ More integration of data. Computer advances permit business to create and manage large 
electronic storehouses of data that cross functional boundaries. 

e Real-time access to knowledge. Today’s computers and software offer the power to collect 
and analyze data and customize reporting in real time for much quicker decision making. 


8. New perspectives on established research methodologies. Businesses are demonstrating a pal- 
pable hunger for breakthrough insights and more effective methods to get them. Businesses 
experiencing rapid technological and social change are looking for researchers who can help 
them keep not just current with the rapid pace of change but in front of it. Older tools and 
methodologies once limited to exploratory or qualitative research are gaining wider acceptance 
in dealing with a broader range of managerial problems. 


To do well in such an environment, you will need to understand how to identify quality information 
and to recognize the solid, reliable research on which your high-risk managerial decisions can be based. 
You will need to know how to conduct such research. Developing these skills requires understanding 
the scientific method as it applies to the decision-making environment. Many students will also need 
to hire research suppliers or write an effective RFP (request for proposal). To facilitate that goal, Ap- 
pendix la, available from the text Online Learning Center, describes how the research industry works. 
Appendix A, at the end of the book, describes how to effectively plan and document research requests 
and proposals. Along with other reference material provided throughout the book, we address your 
needs as information collector, processor, evaluator, and user. 


> Information and Competitive Advantage 


Managers have access to information other than that generated by business research. Understanding the 
relationship between business research and these other information sources—decision support systems 
and business intelligence—is critical for understanding how information drives decisions relating to 
organizational mission, goals, strategies, and tactics. 


Goals 


A local bakery would have different goals than Nabisco, but each likely has goals related to sales 
(membership), market share, return on investment, profitability, customer acquisition, customer satis- 
faction, customer retention, employee productivity, production efficiency, maximization of stock price 
(or owner’s equity), and so on—whether codified in a written plan or detailed only in an entrepreneur’ s 
brain. To assist in making increasingly complex decisions on goals, strategies, and tactics, managers 
turn first to information drawn from the decision support system, combined with that generated by 
business intelligence on competitive and environmental activity. 
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As NTT Communications indicates, “Today, information is everything. It’s the core of your business and you can’t exist without 
it.” NTT uses the private cloud for seamless and secure data access worldwide. 


The need to complete one or many exchanges with its prospective customers, members, or constituents 
drives every organization. No matter how we define an exchange—a purchase, a vote, attendance at 
a function, a donation to a cause—each exchange, along with the strategic and tactical activities de- 
signed to complete it, generates numerous elements of data. If organized for retrieval, collectively these 
data elements constitute a decision support system (DSS). During the last two and one-half decades, 
advances in computer technology made it possible to share this collected transactional data among an 
organization’s decision makers over an intranet or an extranet. 
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Today, sophisticated managers have developed DSSs, where data can be accessed in real time (as 
transactions are completed). Catalog managers (e.g., casual clothing retailer Lands’ End) know exactly 
what tactics generate a transaction from a particular individual within their prospect and customer data- 
bases, as well as just how profitable each customer is to the company and an estimate of that customer’s 
lifetime value to the company. Such managers have a distinct advantage in strategic and tactical plan- 
ning over those without real-time access to transactional data. 


Business Intelligence 


Because no decision exists in a vacuum, the decision maker must have a broad knowledge of the firm’s 
environment. A business intelligence system (BIS) is designed to provide the manager with ongoing 
information about events and trends in the technological, economic, political and legal, demographic, 
cultural, social, and, most critically, competitive arenas. Such information is compiled from a variety 
of sources, as is noted in Exhibit 1-1. 

Often, data from a DSS or BIS stimulate the question “Should we do business research?” In the 
MindWriter example, this might be data collected about laptop problems needing repair; or, for our 
restaurant whose friendliness quotient is changing, it might be customer comments collected by the wait 


staff. 


Strategy 


Strategy is defined as the general approach an organization will follow to achieve its goals. In an 
earlier example, a restaurant was receiving comments that the friendly atmosphere was changing. 
This perception may have been the result of a change in strategy. Perhaps the restaurant decided to 
switch from an atmosphere where patrons were encouraged to linger over their meal (occupying 


>Exhibit 1-1 Some Sources of Business Intelligence 
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Client Perspective of Consultancy Skills Needed to Be an Effective Researcher 


Recently a group of research clients participated in a GreenBook reflect an understanding of the rapidly changing contexts 
blog about researchers consultancy skill needs. Here is what and landscapes that the business faces—nimble, flexible, 
they shared. proactive, and forward thinking. 


e Researchers need to think strategically. Insights 
are ultimately about strategy—spelling out implications 
and recommendations, highlighting opportunities, and 


e Researchers need to challenge decision makers. 
Insights need to be bold, provocative. Business decision 
makers are looking for solutions, direction. 


describing how business decision makers might use e Researchers need to advocate. Researchers need to 
them. tell a story, engage an audience, inspire and impassion 

e Researchers need to be insights- and action- business decision makers to action. They should learn 
specialists. Researchers need to think like business to become as comfortable using the word should as 
decision makers, delivering concrete, call-to-action the word could. Advocacy is fact-supported beliefs, not 
insights. The insights and recommendations need to ambivalence. 


a table for a long period of time while adding incremental revenues with each additional course) 
to a new strategy of turning each table in a shorter time frame by changing food preparation and 
the menu. 

A firm usually implements more than one strategy at a time. With regard to training, one organiza- 
tion might train its data warehouse employees with mostly classroom activities, while another will 
use on-the-job training. Another strategy might describe how an organization handles maintenance 
on its equipment—trigorous periodic maintenance versus maintenance only when equipment breaks 
down. Microsoft recently completed a major corporate restructuring. It decided to tie its 600 managers’ 
compensation, not to sales and profits, but to levels of customer satisfaction as measured by periodic 
customer satisfaction surveys.” 

The discovery of opportunities and problems that influence strategic decisions is often the task of 
the BIS in combination with business research. 


Tactics 


Business research also contributes significantly to the design tactics—those specific, timed ac- 
tivities that execute a strategy. Business research also can be used to help a manager decide which 
of several tactics is likely to successfully execute the desired strategy. In our earlier example, 
our restaurant manager might have changed the menu (marketing tactic) to feature entrées that 
could be prepared faster (operations tactic) and delivered to a table more quickly. The manager 
might also have instituted a new training program (HR tactic) to implement a new zoned, table- 
coverage structure (operations tactic), along with a new sales-incentive program (HR tactic) 
that discouraged the wait staff from making small talk with patrons and rewarded teamwork and 
efficiency. 
All of the above examples demonstrate the purposes of business research: 


¢ To identify and define opportunities and problems. 
¢ To define, monitor, and refine strategies. 
¢ To define, monitor, and refine tactics. 


e To improve our understanding of the various fields of management.’ 
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> Hierarchy of Information-Based 
Decision Makers 


Although not all organizations use business research to help make planning decisions, increasingly the 
successful ones do. Exhibit 1-2 shows an emerging hierarchy of organizations in terms of their use of 
business research. 

In the top tier, organizations see research as the fundamental first step in any venture. They go be- 
yond the tried-and-true methodologies and use creative combinations of research techniques to gain 
deep insights to aid in their sophisticated decision making. Some even develop their own proprietary 
methodologies. These firms may partner with a small group of outside research suppliers that have the 
expertise to use innovative combinations of research methods to address management dilemmas. These 
visionary managers can be found in research firms, service firms, nonprofit organizations, and product 
and service manufacturers and distributors. Minute Maid, the manufacturer that brings us fresh and 
frozen juice-based products, fosters decision making at this level. Its vice president of Consumer and 
Marketing Knowledge is a member of the firm’s highest strategic planning team.* Implementation and 
activation of the research are the critical stages of decision makers in this tier. Design Forum, an archi- 
tectural and graphic design firm specializing in retail design and positioning for such firms as Lexus, 
Dunkin’ Donuts, and McDonald’s, is another firm operating at this level; every recommendation to 
each client is based on data drawn from the use of extensive research. 

In the second tier of the hierarchy are those decision makers that rely periodically on research in- 
formation. They usually turn to business research primarily when they perceive the risk of a particular 
strategy or tactic to be too great to proceed without it. They rely heavily on those methodologies that 
proved themselves in the last several decades of the 20th century—surveys and focus groups—often 
choosing the methodology before fully assessing its appropriateness to the dilemma at hand. This tier is 
occupied by many large, medium, and small organizations of all types. Some of the firms newly arrived 
to this tier are in transition from the base tier. They have realized that failing to collect information prior 
to decision making or failing to extract insight from information that has been collected in their DSS 
puts them at a distinct competitive disadvantage. 


>Exhibit 1-2 Hierarchy of Information-Based Decision Makers 


e Every decision guided by business research. 
e Firms develop proprietary methodologies and are 
innovative in their combination of methodologies. 


Top Tier e Enterprise-wide access to research data and findings. 
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¢ Some decisions based on business research. 
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Pattern Thinking at Yum! Brands 


When PepsiCo spun off its restaurant division into a separate 
corporation, Yum! Brands, Inc., some might have thought that 
the removal of the struggling restaurants from the more popular 
and successful snack foods was an end rather than a beginning. 
But David Novak, saw an opportunity to learn from the best. 
“To take advantage of our unique position of being a brand- 
new public company made up of well-established brands, we did 
a best-practice tour of some of the most successful companies 
around at the time in order to take inspiration from them and borrow 
any good ideas we could find. We visited seven companies in all— 
GE, Walmart, Home Depot, Southwest Airlines, Target, Coke, and 
UPS—and then came back and crystallized what we’d learned.” 
Yum! combined observation with individual depth interviews to 


Pattern thinking is “where you look at what’s working for some- 
one else and apply it to your own situation.“ The technique gener- 
ated more than incremental improvements, it helped Yum! take a 
giant leap forward. Using this approach he and his team identi- 
fied five Dynasty Drivers for Yum! Brands. “These were the things 
that we believed would make us an enduringly great company 
and included: A Company Where Everyone Makes a Difference; 
Customer and Sales Mania; Competitive Brand Differentiation; 
Continuity in People and Process; and Consistency in Results.” 
Novak identified a valuable lesson for researchers. “Pattern 
thinking requires that you keep your eyes open and actively seek 
out new ideas wherever you can find them. And you won’t truly 
have your eyes open unless you have enough humility to admit 


gain insights —patterns—that could be used in the restaurant divi- 
sion. Novak employed a technique he coined as pattern thinking. 


that the best ideas aren’t always going to come from you.” 
www.yum.com 


Finally, the base tier comprises those managers who primarily use instinct and intuition rather than 
research knowledge to facilitate their decisions. These firms may or may not have sophisticated DSSs 
or BISs. They believe themselves to be so close to customers and distribution partners, as well as to 
employees and other stakeholders, that they rarely need business research. When they do collect in- 
formation, they use a limited amount of qualitative research, often in the form of an informal group 
discussion or small number of individual interviews, to confirm their ideas. Especially in the business- 
to-business arena, they often rely on feedback filtered by members of the sales force. Following guide- 
lines for adequate sampling or other procedures of scientific inquiry is not fundamental to this group. 
Larger firms that occupy this tier are influenced as much by organizational culture as by resources. 
Many small companies find themselves in this tier not because of an unwillingness to use business 
research but based on a perception that any more formalized research is too expensive to employ and 
that their resources won’t accommodate this mode of decision making. 

The trends of the past two decades, especially the technology that has been driving research 
methodologies of data collection and dissemination, make it likely that managers who do not prepare 
to advance up the hierarchy will be at a severe competitive disadvantage. Some examples of where 
business collects its data are shown in Exhibit 1-3. 


> The Research Process: A Preview 


Writers usually treat the research study as a sequential process involving several clearly defined steps. 
Exhibit 1-4 models the sequence of the research process. No one claims that research requires comple- 
tion of each step before going to the next. Recycling, circumventing, and skipping occur. Some steps 
are begun out of sequence, some are carried out simultaneously, and some may be omitted. Despite 
these variations, the idea of a sequence is useful for developing a project and for keeping the project 
orderly as it unfolds. 

The research process begins much as the opening vignette suggests. You will notice that the top of 
the model is devoted to understanding the manager’s problem—the management dilemma. A man- 
agement dilemma triggers the need for a decision. For MindWriter, a growing number of complaints 
about postpurchase service started the process. In other situations, a controversy arises, a major com- 
mitment of resources is called for, or conditions in the environment signal the need for a decision. For 


Exhibit 1-3 Where Business Collects Research Information 


Where/How 
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Type of Data 


Transactional 


Observational 


Conversational (Touch 
points) 


Internet Analytics 


Online and in-store purchases 

Online, phone, in-store inquiries 

Warehouse and shipping manifests 

Machine performance 

Online Web visits and in-store shopping trips 
Competitor interactions 

Click-through paths on Web 

In-store customer service interactions 

Stock price valuations 


Biometric measures (e.g., neuromarketing, fMRI, 
PET, eye tracking) 


Surveys, online and in-store intercepts 


Call center interactions 

In-store customer service interactions 
Web chat interactions 

In-store checkout 

Candidate interviews 

Performance reviews 

Exit interviews 

Annual stockholder meetings 
Financial performance presentations 


Listening tours 


Twitter posts 


Facebook posts (company site) 


Blog activity 


Other social media posts or discussions 


Keyword searches 
Click analysis 
Google+ 


Data Source 

Customer 

Potential customer, customer 
Logistic partners, employee 
Machine data log 

Customer, employee 
Customer 

Potential customer, customer 
Customer, employee 
Investors 


Potential customer, customer, employee 


Potential customer, customer, employee 


Customer, employee 

Customer, employee 

Customer, employee 

Customer, employee 

Potential employee 

Employee 

Employee 

Investor 

Financial analyst, institutional investor 


Customer, supplier, logistic partner, 
employee, decision influencer 


Customer, employee, competitor, trade 
associations, distributor 


Customer, employee, trade associations, 
distributor 


Customer, employee, competitor, trade 
associations, distributor 


Customer, employee, competitor, trade 
associations, distributor 


Potential customer, customer 
Potential customer, customer 


Potential customer, customer 


Our interviews and research for this edition, revealed several sources of research data. This table is adapted from that research and 
author experience as well as from material by Cynthia Clark, “S Ways to Learn What Customers Aren’t Telling You,” 1to1 Magazine, 
March 5, 2012, accessed March 8, 2012 (http://www. 1to1media.com/view.aspx?docid=33464) and “Harness the Conversation: 
Business in Today’s Social World,” Cvent, accessed March 8, 2012 (http://www.cvent.com/en/sem/business-in-todays-social-world- 


survey-ebook.shtml). 


MindWriter, the critical event could have been the introduction by a competitor of new technology 
that would revolutionize the battery life of laptops. Such events cause managers to reconsider their 
purposes or objectives, define a problem for solution, or develop strategies and tactics for solutions 


they have identified. 


In every chapter, we will refer to this model as we discuss each step in the process. Our discussion 
of the questions that guide project planning and data gathering is also incorporated into the various 
elements of the model. In the chapters that follow, we discuss scientific research procedures and 
ethical conduct by showing their application to the pragmatic problems of managers. At a minimum, 
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>Exhibit 1-4 The Research Process 


Chapters 2-5 


Exploration Exploration 


Appendix A 


Research Design 
Strategy 


(type, purpose, time frame, scope, environment) Chapters 6-14 


Data Collection Sampling 
Design Design 


Chapter 15 


Data Collection & Preparation 


Chapters 16-18 


Research Reporting 
Chapters 19-20 


Management 
Decision 


our objective is to make you a more intelligent consumer of research products prepared by others 
(see Appendix la, available from the text Online Learning Center) as well as to enable you to perform 
quality research for your own decisions and those to whom you report. 

Exhibit 1-4 is an important organizing tool because it provides a framework for introducing how 
each process module is designed, connected to other modules, and then executed. Thus, it organizes 
the book. 
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Is Research Always Problem-Solving Based? 


Researchers often are asked to respond to “problems” that managers needed to solve. Applied research 
has a practical problem-solving emphasis. Whether the problem is negative, like rectifying an inventory 
system that is resulting in lost sales, or an opportunity to increase stockholder wealth through acquiring 
another firm, problem solving is prevalent. 

The problem-solving nature of applied research means it is conducted to reveal answers to specific 
questions related to action, performance, or policy needs. Pure research or basic research is also 
problem-solving based, but in a different sense. It aims to solve perplexing questions or obtain new 
knowledge of an experimental or theoretical nature that has little direct or immediate impact on action, 
performance, or policy decisions. Basic research in the business arena might involve a researcher who 
is studying the results of the use of coupons versus rebates as demand stimulation tactics, but not in a 
specific instance or in relation to a specific client’s product. In another pure research scenario, research- 
ers might study the influence on productivity of compensation systems that pay by piece-work versus 
salary-plus-bonus structures. Thus, both applied and pure research are problem-solving based, but ap- 
plied research is directed much more to making immediate managerial decisions. 

In answer to the question posed at the beginning of this section, Is research always problem-solving 
based? the answer is yes. Whether the typology is applied or pure, simple or complex, all research 
should provide an answer to some question. If managers always knew what was causing problems or 
offering opportunities in their realm of responsibility, there would be little need for applied research or 
basic research; intuition would be all that was necessary to make quality decisions. 


> What Is Good Research? 


Good research generates dependable data that are derived by professionally conducted practices and 
that can be used reliably for decision making. In contrast, poor research is carelessly planned and 
conducted, resulting in data that a manager can’t use to reduce his or her decision-making risks. Good 
research follows the standards of the scientific method: systematic, empirically based procedures for 
generating replicable research. 

We list several defining characteristics of the scientific method in Exhibit 1-5 and discuss below the 
managerial dimensions of each. 


1. Purpose clearly defined. The purpose of the business research—the problem involved or the 
decision to be made—should be clearly defined and sharply delineated in terms as unambigu- 
ous as possible. Getting this in writing is valuable even in instances in which the same person 
serves as researcher and decision maker. The statement of the decision problem should include 
its scope, its limitations, and the precise meanings of all words and terms significant to the 
research. Failure of the researcher to do this adequately may raise legitimate doubts in the 
minds of research report readers as to whether the researcher has sufficient understanding of 
the problem to make a sound proposal attacking it. 


2. Research process detailed. The research procedures used should be described in sufficient 
detail to permit another researcher to repeat the research. This includes the steps to acquire 
participants, informed consent, sampling methods and representativeness, and data gathering 
procedures. Except when secrecy is imposed, research reports should reveal with candor the 
sources of data and the means by which they were obtained. Omission of significant procedural 
details makes it difficult or impossible to estimate the validity and reliability of the data and 
justifiably weakens the confidence of the reader in the research itself as well as any recommen- 
dations based on the research. 


3. Research design thoroughly planned. The procedural design of the research, and its choice 
among competing designs, should be clearly described and carefully planned to yield results 
that are as objective as possible. A survey of opinions or recollections ought not to be used 
when more reliable evidence is available from documentary sources or by direct observation. 
Bibliographic searches should be as thorough and complete as possible. Experiments should 
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>Exhibit 1-5 What Actions Guarantee Good Business Research? 


What a Manager Should Look For in Research Done by Others or 


Characteristics of Resear Include in Self-Directed Research Chapter 


Purpose clearly defined ¢ Researcher distinguishes between symptom of organization’s 4,5 
problem, the manager’s perception of the problem, and the 
research problem. 


Research design thoroughly planned Exploratory procedures are outlined with constructs defined. 3, 4, 5, 6-14 


¢ Sample unit is clearly described along with sampling methodology. 


Data collection procedures are selected and designed. 


Limitations frankly revealed ¢ Desired procedure is compared with actual procedure in report. 6, 14, 15, 19, 


¢ Desired sample is compared with actual sample in the report. 20 


¢ Impact on findings and conclusions is detailed. 


Findings presented unambiguously ¢ Findings are clearly presented in words, tables, and graphs. 15-20 


¢ Findings are logically organized to facilitate reaching a decision 
about the manager’s problem. 


¢ Executive summary of conclusions is outlined. 


¢ Detailed table of contents is tied to the conclusions and findings 


presentation. 
Researcher’s experience reflected e Researcher provides experience/credentials with report. 19, 20 


have satisfactory controls, reducing threats to internal validity and enhancing the probability of 
external validity (generalizability). Direct observations should be recorded as soon as possible 
after the event. Efforts should be made to minimize the influence of personal bias in selecting 
and recording data. 


4. High ethical standards applied. Researchers often work independently and have significant 
latitude in designing and executing projects. A research design that includes safeguards against 
causing mental or physical harm to participants and makes data integrity a first priority should 
be highly valued. Ethical issues in research reflect important moral concerns about the practice 
of responsible behavior in society. 

Researchers frequently find themselves precariously balancing the rights of their subjects 
against the scientific dictates of their chosen method. When this occurs, they have a responsi- 
bility to guard the welfare of the participants in the studies and also the organizations to which 
they belong, their clients, their colleagues, and themselves. Careful consideration must be given 
to those research situations in which there is a possibility for physical or psychological harm, 
exploitation, invasion of privacy, and/or loss of dignity. The research need must be weighed 
against the potential for these adverse effects. Typically, you can redesign a study, but some- 
times you cannot. The researcher should be prepared for this dilemma. 
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Tom H. C. Anderson - Next Gen Market Research 


More Than Market Research - Gain The Information Advantage 


FRONT PAGE ARCHIVES ABOUT SUBSCRIBE IN A READER 


TG GEN 
WNT] RESEARCH, 


application/rss+xml 


Next Gen Market Research Top Blogs 


© ADDTHIS wf 90 a. 


ABOUT TOM H. C. ANDERSON 


Top 
GEN 
AFFILIATIONS MARKET 
ODINTEXT =o 
Anderson 


The Most Influential in Research - The ‘Who’s Who’ of Future 
Research Next Gen Market Researcher 
The NGMR badges serve as a digital badge of honor as well as a sign of our 

belief in embracing new technology while not ignoring the importance of the 


sound methodology our industry was founded on. cool. NGMR FRIENDS 


Top 10 Influencers 


Fit & @ de 


The Sncial Influencers 


ato 


On Social Media 


Owner 


Blog Name 


COOL NGMR EVENTS 


Merlien Institute 


AUDIENCE 


MEASUREMENT 7.0 


Sean Copeland 


17 


>picorofile 


Keeping abreast of new methodologies and techniques is a never-ending process for a strong researcher. Following research- 
related blogs, following and participating in LinkedIn groups, and attending conferences are critical to this process. The 
Next Generation Market Research blog and LinkedIn group managed by Tom Anderson, founder and managing partner of 
Anderson Analytics, is a prime example. Note that Anderson provides links to many other research-related blogs from his own 


at www.tomhcanderson.com/next-gen-market-research-top-blogs. 


5. Limitations frankly revealed. The researcher should report, with complete frankness, flaws in 


procedural design and estimate their effect on the findings. There are very few perfect research 


designs. Some of the imperfections may have little effect on the validity and reliability of the 
data; others may invalidate them entirely. A competent researcher should be sensitive to the 
effects of imperfect design. The researcher’s experience in analyzing data should provide a 


basis for estimating the influence of design flaws. As a decision maker, you should question the 


value of research about which no limitations are reported. 


6. Adequate analysis for decision maker’s needs. Analysis of the data should be extensive enough 


to reveal its significance, what managers call insights. The methods of analysis used should 
be appropriate. The extent to which this criterion is met is frequently a good measure of the 
competence of the researcher. Adequate analysis of the data is the most difficult phase of 

research for the novice. The validity and reliability of data should be checked carefully. The 
data should be classified in ways that assist the researcher in reaching pertinent conclusions 


and clearly reveal the findings that have led to those conclusions. When statistical methods are 


used, appropriate descriptive and inferential techniques should be chosen, the probability of 
error should be estimated, and the criteria of statistical significance applied. 


7. Findings presented unambiguously. Some evidence of the competence and integrity of the 
researcher may be found in the report itself. For example, language that is restrained, clear, 
and precise; assertions that are carefully drawn and hedged with appropriate reservations; 
and an apparent effort to achieve maximum objectivity tend to leave a favorable impression 
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of the researcher with the decision maker. Generalizations that outrun the statistical findings 
or other evidence on which they are based, exaggerations, and unnecessary verbiage tend to 
leave an unfavorable impression. Such reports are not valuable to managers wading through 
the minefields of organizational decision making. Presentation of data should be comprehen- 
sive, reasonably interpreted, easily understood by the decision maker, and organized so that 
the decision maker can readily locate critical findings. 


8. Conclusions justified. Conclusions should be limited to those for which the data provide an 
adequate basis. Researchers are often tempted to broaden the basis of induction by including 
personal experiences and their interpretations—data not subject to the controls under which the 
research was conducted. Equally undesirable is the all-too-frequent practice of drawing conclu- 
sions from a study of a limited population and applying them universally. Researchers also may 
be tempted to rely too heavily on data collected in a prior study and use it in the interpretation 
of a new study. Such practice sometimes occurs among research specialists who confine their 
work to clients in a small industry. These actions tend to decrease the objectivity of the research 
and weaken readers’ confidence in the findings. Good researchers always specify the conditions 
under which their conclusions seem to be valid. 


9. Researcher’s experience reflected. Greater confidence in the research is warranted if the 
researcher is experienced, has a good reputation in research, and is a person of integrity. Were it 
possible for the reader of a research report to obtain sufficient information about the researcher, 
this criterion perhaps would be one of the best bases for judging the degree of confidence a 
piece of research warrants and the value of any decision based upon it. For this reason the 
research report should contain information about the qualifications of the researcher. 


Good business research has an inherent value only to the extent that it helps management make better 
decisions to achieve organizational goals. Interesting information about consumers, employees, com- 
petitors, or the environment might be pleasant to have, but its value is limited if the information can- 
not be applied to a critical decision. If a study does not help management select more effective, more 
efficient, less risky, or more profitable alternatives than otherwise would be the case, its use should 
be questioned. Alternatively, management may have insufficient resources (time, money, or skill) to 
conduct an appropriate study or may face a low level of risk associated with the decision at hand. In 
these situations, it is valid to avoid business research and its associated costs in time and money. Busi- 
ness research finds its justification in the contribution it makes to the decision maker’s task and to the 
bottom line. 


> A Glimpse at Four Research Studies 


From each of the following illustrations of management dilemmas, we can abstract the essence of 
research. How is it carried out? What can it do? What should it not be expected to do? As you read 
the four cases, be thinking about the possible range of situations for conducting research, and try 
answering these questions: (1) What is the decision-making dilemma facing the manager? (2) What 
must the researcher accomplish? 


ClassicToys 


You work for ClassicToys, a corporation that is considering the acquisition of a toy manufacturer. 
The senior vice president for development asks you to head a task force to investigate six companies 
that are potential candidates. You assemble a team composed of representatives from the relevant 
functional areas. Pertinent data are collected from public sources because of the sensitive nature of the 
project. You examine all of the following: company annual reports; articles in business journals, trade 
magazines, and newspapers; financial analysts’ assessments; and company advertisements. The team 
members then develop summary profiles of candidate firms based on the characteristics gleaned from 
the sources. The final report highlights the opportunities and problems that acquisition of the target firm 
would bring to all areas of the business. 
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MedImage 


You are the business manager for MedImage, a large group of physicians specializing in diagnostic 
imaging (MRI, nuclear, tomography, and ultrasound). A prominent health insurance organization has 
contacted you to promote a new cost-containment program. The doctors’ committee to which you will 
make a recommendation will have a narrow enrollment window for their decision. If they choose to join, 
they will agree to a reduced fee schedule in exchange for easier filing procedures, quicker reimburse- 
ment, and listing on a physicians’ referral network. If they decline, they will continue to deal with their 
patients and the insurance carrier in the current manner. You begin your investigation by mining data 
from patient files to learn how many are using this carrier, frequency of care visits, complexity of filings, 
and so on. You then consult insurance industry data to discover how many potential patients in your 
area use this care plan, or similar care plans with alternative insurance carriers, and the likelihood of a 
patient choosing or switching doctors to find one that subscribes to the proposed program. You attempt 
to confirm your data with information from professional and association journals. Based on this informa- 
tion, you develop a profile that details the number of patients, overhead, and potential revenue realized 
by choosing to join the plan. 


MoreCoatings 


MoreCoatings, a paint manufacturer, is having trouble maintaining profits. The owner believes inven- 
tory management is a weak area of the company’s operations. In this industry, many paint colors, types 
of paint, and container sizes make it easy for a firm to accumulate large inventories and still be unable 
to fill customer orders. You look into the present warehousing and shipping operations and find exces- 
sive sales losses and delivery delays because of out-of-stock conditions. An informal poll of customers 
confirms your impression. You suspect the present inventory database and reporting system do not 
provide prompt, usable information needed for appropriate production decisions. 

Based on this supposition, you familiarize yourself with the latest inventory management tech- 
niques. You ask the warehouse manager to take an inventory, and you review the incoming orders 
for the last year. In addition, the owner shows you the production runs of the last year and his method 
for assessing the need for a particular color or paint type. By modeling the last year of business using 
production, order, and inventory management techniques, you choose the method that provides the best 
theoretical profit. You run a pilot line using the new control methodology. After two months, the data 
show a much lower inventory and a higher order fulfillment rate. You recommend that the owner adopt 
the new inventory method. 


York College 


You work for York College’s alumni association. It is eager to develop closer ties with its aging alumni 
to provide strong stimuli to encourage increased donations and to induce older, nontraditional students 
to return to supplement enrollment. The president’s office is considering starting a retirement com- 
munity geared toward university alumni and asks your association to assess the attractiveness of the 
proposal from an alumni viewpoint. Your director asks you to divide the study into four parts. 


Phase 1 


First you are to report on the number of alumni who are in the appropriate age bracket, the rate of new 
entries per year, and the actuarial statistics for the group. This information allows the director to assess 
whether the project is worth continuing. 


Phase 2 


Your early results reveal a sufficient number of alumni to make the project feasible. The next step in 
the study is to describe the social and economic characteristics of the target alumni group. You review 
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gift statistics, analyze job titles, and assess home location and values. In addition, you review files from 
the last five years to see how alumni responded when they were asked about their income bracket. You 
are able to describe the alumni group for your director when you finish. 


Phase 3 


It is evident that the target alumni can easily afford a retirement community as proposed. The third 
phase of the study is to explain the characteristics of alumni who would be interested in a university- 
related retirement community. For this phase, you engage the American Association of Retired Persons 
(AARP) and a retirement community developer. In addition, you search for information on senior 
citizens from the federal government. From the developer you learn what characteristics of retirement 
community planning and construction are most attractive to retirees. From the AARP you learn about 
the main services and features that potential retirees look for in a retirement community. From govern- 
ment publications you become familiar with existing regulations and recommendations for operating 
retirement communities and uncover a full range of descriptive information on the typical retirement 
community dweller. You make an extensive report to both the alumni director and the university presi- 
dent. The report covers the number of eligible alumni, their social and economic standings, and the 
characteristics of those who would be attracted by the retirement community. 


Phase 4 


The report excites the college president. She asks for one additional phase to be completed. She needs 
to predict the number of alumni who would be attracted to the project so that she can adequately plan 
the size of the community. At this point, you call on the business school’s research methods class for 
help in designing a questionnaire for the alumni. By providing telephones and funding, you arrange 
for the class to conduct a survey among a random sample of the eligible alumni population. In addi- 
tion, you have the class devise a second questionnaire for alumni who will become eligible in the next 
10 years. Using the data collected, you can predict the initial demand for the community and estimate 
the growth in demand over the next 10 years. You submit your final report to the director and the 
president. 


What Dilemma Does the Manager Face? 


The manager’s predicament is fairly well defined in the four cases. Let’s see how carefully you read 
and understood them. In the ClassicToys study, the manager, the senior vice president for develop- 
ment, must make a proposal to the president or possibly the board of directors about whether to acquire 
a toy manufacturer and, if one is to be acquired, which one of the six under consideration is the best 
candidate. In MedImage, the physicians in the group must decide whether to join the proposed man- 
aged health care plan of one of their primary insurers. In the MoreCoatings study, the owner of the 
paint manufacturer must decide whether to implement a new inventory management system. At York 
College, the president must propose to the board of directors whether to fund the development of a 
retirement community. How did you do? If you didn’t come to these same conclusions, reread the cases 
before proceeding to catch what you missed. 

In real life, management dilemmas are not always so clearly defined. In the MoreCoatings study, 
rather than pinpointing the problem as one of inventory management, the paint manufacturer’s owner 
could have faced several issues: (1) a strike by the teamsters impacting inventory delivery to retail and 
wholesale customers; (2) the development of a new paint formula offering superior coverage but re- 
quiring a relatively scarce ingredient to manufacture, thereby affecting production rates; (3) a fire that 
destroyed the primary loading dock of the main shipping warehouse in the Midwest; (4) the simultane- 
ous occurrence of all three events. As the research process begins with a manager’s decision-making 
task, accurately defining the dilemma is paramount but often difficult. We outline the research process 
that begins this activity at the end of this chapter and address it in detail in Chapter 4. 
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The Types of Research Studies Represented 
by the Four Examples 


All four studies qualify as applied research and can be classified as reporting, descriptive, explanatory, 
or predictive. 


Reporting 


At the most elementary level, a reporting study provides a summation of data, often recasting data to 
achieve a deeper understanding or to generate statistics for comparison. The task may be quite simple 
and the data readily available. At other times, the information may be difficult to find. A reporting study 
calls for knowledge and skill with information sources and gatekeepers of information sources. Such a 
study usually requires little inference or conclusion drawing. In the ClassicToys study, the researcher 
needs to know what information should be evaluated in order to value a company. In the study of man- 
agement, this knowledge would be acquired primarily in courses in financial management, accounting, 
and marketing. Knowing the type of information needed, the researcher in the ClassicToys study iden- 
tifies sources of information, such as trade press articles and annual reports. Because of the possible 
effect of the toy manufacturer evaluation on the stock prices of the conglomerate instigating the study 
and each toy company, only public sources are used. Other reporting studies of a less sensitive nature 
might have the researcher interviewing source gatekeepers. In the York College study, for example, 
interviewing the director of local retirement facilities might have revealed other sources to include in 
the search. Such an expert is considered a gatekeeper. Early in your career, identifying gatekeepers for 
your firm and industry is critical to success as a manager. 

Purists might argue that reporting studies do not qualify as research, although such carefully gath- 
ered data can have great value. A research design does not have to be complex and require inferences 
for a project to be called research. In the early part of your career, you will likely be asked to perform 
a number of reporting studies. Many managers consider the execution of such studies an excellent way 
for new employees to become familiar with their employer and its industry. 


Descriptive 


A descriptive study tries to discover answers to the questions who, what, when, where, and, some- 
times, how. The researcher attempts to describe or define a subject, often by creating a profile of a 
group of problems, people, or events. Such studies may involve the collection of data and the creation 
of a distribution of the number of times the researcher observes a single event or characteristic (known 
as a research variable), or they may involve relating the interaction of two or more variables. In Med- 
Image, the researcher must present data that reveal who is affiliated with the insurer, who uses managed 
health care programs (both doctors and patients), the general trends in the use of imaging technology 
in diagnosing illness or injury severity, and the relationship of patient characteristics, doctor referrals, 
and technology-use patterns. 

Descriptive studies may or may not have the potential for drawing powerful inferences. Organi- 
zations that maintain databases of their employees, customers, and suppliers already have significant 
data to conduct descriptive studies using internal information. Yet many firms that have such data 
files do not mine them regularly for the decision-making insight they might provide. In the open- 
ing vignette, Myra Wines could mine numerous company databases for insight into the nature and 
number of service-related problems arising after purchase and, similarly, for information about 
product use inquiries. A database generated by warranty registration cards could reveal significant 
data concerning purchaser characteristics, as well as purchase location and product use behavior. A 
descriptive study, however, does not explain why an event has occurred or why the variables interact 
the way they do. 

The descriptive study is popular in research because of its versatility across management disci- 
plines. In not-for-profit corporations and other organizations, descriptive investigations have a broad 
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appeal to the administrator and policy analyst for planning, monitoring, and evaluating. In this context, 
how questions address issues such as quantity, cost, efficiency, effectiveness, and adequacy.° 


Explanatory 


Academics debate the relationship between the next two types of studies, explanatory and predic- 
tive, in terms of which precedes the other. Both types of research are grounded in theory, and 
theory is created to answer why and how questions. For our purposes, an explanatory study goes 
beyond description and attempts to explain the reasons for the phenomenon that the descriptive 
study only observed. Research that studies the relationship between two or more variables is also 
referred to as a correlational study. The researcher uses theories or at least hypotheses to account 
for the forces that caused a certain phenomenon to occur. In MoreCoatings, believing the prob- 
lem with paint stockouts is the result of inventory management, the owner asks the researcher to 
detail warehousing and shipping processes. It would be a descriptive study if it had stopped here. 
But if problems in the processes could be linked with sales losses due to an inability to make 
timely deliveries to retail or wholesale customers, then an explanatory study would emerge. The re- 
searcher tests this hypothesis by modeling the last year of business using the relationships between 
processes and results. 


Predictive 


If we can provide a plausible explanation for an event after it has occurred, it is desirable to be able to 
predict when and in what situations the event will occur. A predictive study, the fourth type, is just as 
rooted in theory as explanation. NATA, a national trade association for the aviation industry, may be 
interested in explaining the radiation risks from the sun and stars for flight crews and passengers. The 
variables might include altitude, proximity of air routes to the poles, time of year, and aircraft shield- 
ing. Perhaps the relations among the four variables explain the radiation risk variable. This type of 
study often calls for a high order of inference making. Why, for example, would a flight at a specified 
altitude at one time of year not produce so great a radiation risk to the airliner’s occupants as the same 
flight in another season? The answer to such a question would be valuable in planning air routes. It 
also would contribute to the development of a better theory of the phenomenon. In business research, 
prediction is found in studies conducted to evaluate specific courses of action or to forecast current and 
future values. 

The researcher is asked to predict the success of the proposed retirement facility for alumni for 
York College based on the number of applicants for residency the project will attract. This prediction 
will be based on the explanatory hypothesis that alumni frequent programs and projects sponsored by 
the institution because of an association they maintain between their college experience and images of 
youthfulness and mental and physical stimulation. 

Finally, we would like to be able to control a phenomenon once we can explain and predict it. 
Being able to replicate a scenario and dictate a particular outcome is the objective of control. In the 
York College study, if we assume that the college proceeds with its retirement community and enjoys 
the predicted success, the president will find it attractive to be able to build a similar facility to serve 
another group of alumni and duplicate that success. 

Control is a logical outcome of prediction. The complexity of the phenomenon and the adequacy 
of the prediction theory, however, largely decide success in a control study. At York College, if a 
control study were done of the various promotional approaches used with alumni to stimulate im- 
ages of youthfulness, the promotional tactics that drew the largest number of alumni applications for 
residency could be identified. Once known, this knowledge could be used successfully with different 
groups of alumni only if the researcher could account for and control all other variables influencing 
applications. 

Any of the four types of studies—reporting, descriptive, explanatory, or predictive—can properly 
be called research. We also can conclude from the various examples that research is a systematic 
inquiry aimed at providing information to solve managerial problems. 
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ET 


Research is any organized inquiry carried out to provide 
information for solving problems. This includes reporting, 
descriptive, explanatory, and predictive studies. We empha- 
size the last three in this book. Business research is a sys- 
tematic inquiry that provides information to guide decisions. 
More specifically, it is a process of determining, acquiring, 
analyzing and synthesizing, and disseminating relevant data, 
information, and insights to decision makers in ways that 
mobilize the organization to take appropriate actions that, in 
turn, maximize performance. If organized for retrieval, data 
collected from the day-to-day operations of the organiza- 
tion constitute a decision support system (DSS). A business 
intelligence system (BIS) is designed to provide the manager 
with ongoing information about events and trends in the 
technological, economic, political and legal, demographic, 
cultural, social, and, most critically, competitive arenas. 
Research studies are used to supplement DSS and BIS. 


The managers of tomorrow will need to know more than any 
managers in history. Business research will be a major con- 
tributor to that Knowledge. Managers will find Knowledge of 
research methods to be of value in many strategic and tacti- 
cal situations. They may need to conduct research either for 
themselves or for others. As buyers of research services, 
managers will need to be able to judge research quality. 
Finally, they may become research specialists themselves. 
Not all managers have established research as a prior- 
ity in their process of decision making. Consequently, a 
hierarchy of research-based decision makers is emerging. 
The top tier contains those managers who use research as 
a fundamental step in all decisions and who use creative vi- 
sion to establish proprietary methodologies. The middle tier 
includes those managers who occasionally turn to research 
but only rely on the tried-and-true methods. The bottom 
tier is those managers who by choice or economic circum- 
stance choose to rely on intuition and judgment rather than 
business research. 


The research process is a model for the development and 
interpretation of research studies. Although many research- 
ers perceive the research study as a sequential process 


involving several clearly defined steps, no one claims that re- 
search requires completion of each step before going to the 
next. Recycling, circumventing, and skipping occur. Some 
steps are begun out of sequence, some are carried out 
simultaneously, and some may be omitted. Despite these 
variations, the idea of a sequence is useful for developing a 
project and for keeping the project orderly as it unfolds. 


What characterizes good research? Generally, one expects 
good research to be purposeful with a clearly defined focus 
and plausible goals; with defensible, ethical, and repeatable 
procedures; and with evidence of objectivity. The reporting 
of procedures —their strengths and weaknesses — should 
be complete and honest. Appropriate analytical techniques 
should be used; conclusions drawn should be limited to 
those clearly justified by the findings; and reports of find- 
ings and conclusions should be clearly presented and 
professional in tone, language, and appearance. Managers 
should always choose a researcher who has an estab- 
lished reputation for quality work. The research objective 
and its benefits should be weighed against potentially 
adverse effects. 


Research is any organized inquiry carried out to provide 
information for solving problems. This includes reporting, 
descriptive, explanatory, and predictive studies. Reporting 
studies provide a summation of data, often recasting data 
to achieve a deeper understanding or to generate statis- 
tics for comparison. A descriptive study tries to discover 
answers to the questions who, what, when, where, and, 
sometimes, how. An explanatory study attempts to explain 
the reasons for the phenomenon that the descriptive study 
only observed. A predictive study attempts to predict when 
and in what situations an event will occur. Studies may 
also be described as applied research or basic research. 
Applied research applies research to discovering solutions 
for immediate problems or opportunities. Basic (or pure) 
research aims to solve perplexing questions or obtain new 
knowledge of an experimental or theoretical nature that has 
little direct or immediate impact on action, performance, or 
policy decisions. 


pa CIS 


applied research 15 explanatory study 22 research variable 21 


business intelligence system (BIS) 9 management dilemma 6 return on investment (ROI) 6 


business research 4 predictive study 22 scientific method 15 


control 22 pure research (basic research) 15 strategy 9 


decision support system (DSS) 8 reporting study 21 tactics 10 


descriptive study 21 research process 12 
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Terms in Review 
1 What is business research? Why should there be any 
question about the definition of research? 


2 What is the difference between applied research and basic 
or pure research? Use a decision about how a salesperson 
is to be paid, by commission or salary, and describe the 
question that would guide applied research versus the 
question that would guide pure research. 


3 Distinguish between an explanatory and predictive re- 
search study. 


4 Distinguish between a reporting study and a descriptive study. 


Making Research Decisions 

5 Asales force manager needs to have information in order 
to decide whether to create a custom motivation program 
or purchase one offered by a consulting firm. What are the 
dilemmas the manager faces in selecting either of these 
alternatives? 

6 Toyota had a major problem with unexplained acceleration in 
several of its top models in 2010. It closed down production 
and stopped sales of multiple models. What types of research 
might Toyota have conducted to make these decisions? 


7 You have received a business research report done by 
a consultant for your firm, a life insurance company. The 
study is a survey of customer satisfaction based on a 
sample of 600. You are asked to comment on its quality. 
What will you look for? 


8 As area sales manager for a company manufacturing and 
marketing outboard engines, you have been assigned the 


responsibility of conducting a research study to estimate 
the sales potential of your products in the domestic (U.S. 
or Canadian) market. Discuss key issues and concerns 
arising from the fact that you, the manager, are also the 
researcher. 


Bringing Research to Life 
9 What evidence is presented in the Bringing Research to Life 
vignette of efforts to understand the management dilemma? 


From Concept to Practice 
10 Apply the principles in Exhibit 1-4 to the research scenario 
in question 8. 


From the Headlines 


11 Kathy Lee Berggren, a professor of oral communication 
at Cornell University, indicates “a lot of my students re- 
ally [only] scratch the surface with the type of research 
they’re doing.” According to Andy Guess, at Inside Higher 
Ed, “Just because students walk in the door as ‘digital 
natives’, doesn’t mean they’re equipped to handle the 
heavy lifting of digital databases and proprietary search 
engines that comprise the bulk of modern, online re- 
search techniques.” Students erroneously think a Google 
search is research. 

As you read through the reasons that should 
stimulate your interest in studying research methods or 
evaluate the nine factors that guarantee good research, 
what actions do you propose to narrow the gap between 
students’ research competence and what’s required of a 
modern college graduate about to become a manager? 


OT 


ee Data Development Inc. 


HeroBuilders.com 


Ohio Lottery: Innovative Research Design Drives 
Winning 


*You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 
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You'll find the following appendix available from the Online Learning Center to supplement the content of this chapter: 


Appendix 1a: How the Research Industry Works. 


chapter 2 


learningobjectives 
After reading this chapter, you should understand. . . 


What issues are covered in research ethics. 


The goal of “no harm” for all research activities and what constitutes “no harm” for participant, researcher, 
and research sponsor. 


The differing ethical dilemmas and responsibilities of researchers, sponsors, and research assistants. 


The role of ethical codes of conduct in professional associations. 


6 6 Today, it would be remiss to say that the privacy 
profession is anything but flourishing. Companies 
are increasingly hiring privacy officers and even 
elevating them to C-suite positions; the European 
Commission has proposed a statute in its amended 


data protection framework that would require data 
protection officers at certain organizations, and at 
the International Association of Privacy Professionals 


(IAPP) membership recently hit 10,000 worldwide. 9 9 
Angelique Carson, CIPP/US, 
International Association of Privacy Professionals 


bringingresearcnhtolife 


“How did the proposal meeting go with MicroPeripheral 
this morning,” inquired Sara as Jason dropped into a 
chair across from Sara’s desk. 

“Tt didn’t,” flatly stated Jason. 

“Well, we had a good proposal, but we can’t win 
every contract,’ commiserated Sara. “Do we know 
why?” 

“Oh, we could have had the contract,” claimed 
Jason. “I just decided the contract wasn’t right for 
Henry and Associates.” 

“Now you’ve got my attention,’ said Sara, 
leaning forward. “That would have been a small 
piece of business now, but being a research supplier 
to such a large computer peripheral manufacturer 
showed definite possibilities for a lucrative long-term 
relationship. What went wrong?” 

“According to its president, Bill Henderson,” 
explained Jason as he and Sara walked back to Jason’s 
office, “MicroPeripheral (MP) has taken a near-lead 
position in peripherals for laptop computing, but 
peripherals are volatile. Peripherals grow smaller every 
month and have to be sold more cheaply. Henderson 
needed a detailed market report that MP could very 
well afford to pay for, but he really wanted something 
we couldn’t deliver.” 

“The proposal we developed was for a detailed 
market study,” puzzled Sara. 

“Yes, but he had in mind an entirely different study 
than we proposed. He proposed hiring a headhunter to 
set up interviews for a mythical senior diversification 
manager position within a mythical company—he 
called it a disguised study. He wanted us to provide 
the focus group facility for the interviews so that we 


could use our equipment to digitally record every 
interview.” 

“T fail to see how this would have given him the 
market data he needed,” stated Sara, now clearly 
confused. 
another CEO had 
tried this ploy and been able to attract competitors’ 


“According to Henderson, 


employees to be interviewed. Every interview added 
greatly to the firm’s understanding of the competition 
and the market. But then they hit the jackpot. One of the 
candidates was a key exec from the major competitor. 
On the basis of that interview and the information the 
duped executive innocently revealed, the company 
decided to shut down its California production line 
and open production in Mexico for a smaller, faster, 
cheaper version of its main product. The advance 
notice permitted the firm to steal significant market 
share.” 

“He had the gall to suggest we participate in this 
fraud?” 

“Henderson assured me that it wasn’t illegal, but I 
didn’t stick around to hear more. I literally grabbed our 
proposal from the desk top—the one with the Henry 
and Associates logo boldly marked on the front—and 
walked out.” 

“So I imagine you’ ve been on the phone since you 
returned,” surmised Sara. 

“T called every single research firm whose logo was 
visible in the array of proposals on Henderson’s desk. 
They were all most appreciative,” said Jason, smiling 
for the first time. 

“And it’s likely,” smiled Sara, “that Henderson will 
soon find all his other proposals will be withdrawn.” 
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Are Research Ethics? 


As in other aspects of business, all parties in research should exhibit ethical behavior. Ethics are norms 
or standards of behavior that guide moral choices about our behavior and our relationships with oth- 
ers. The goal of ethics in research is to ensure that no one is harmed or suffers adverse consequences 
from research activities. This objective is usually achieved. However, unethical activities are pervasive 
and include violating nondisclosure agreements, breaking participant confidentiality, misrepresenting 
results, deceiving people, using invoicing irregularities, avoiding legal liability, and more. 

The recognition of ethics as a problem for economic organizations is repeatedly revealed in sur- 
veys. Despite an increase in awareness of formal ethics programs and the presence of written ethical 
codes of conduct (83 percent), one survey reports that although 52 percent of U.S. workers claim to 
have observed at least one type of ethical misconduct, only 55 percent of those observing misconduct 
reported the violation (a decline of 10 percent from a similar study done two years before). Strong- 
ethical-culture organizations were 1.6 times as likely to report incidents as weak-ethical-culture 
organizations. Such lack of action raises questions about the effectiveness of codes of conduct and 
reporting systems.' 

There is no single approach to ethics. Advocating strict adherence to a set of laws is difficult because 
of the unforeseen constraint put on researchers. Because of Germany’s war history, for example, the 
government forbids many types of medical research. Consequently, the German people do not benefit 
from many advances in biotechnology and may have restricted access to genetically altered drugs in the 
future. Alternatively, relying on each individual’s personal sense of morality is equally problematic. 
Consider the clash between those who believe death is deliverance from a life of suffering and those 
who value life to the point of preserving it indefinitely through mechanical means. Each value system 
claims superior knowledge of moral correctness. 

Clearly, a middle ground between being completely code governed or relying on ethical relativism 
is necessary. The foundation for that middle ground is an emerging consensus on ethical standards for 
researchers. Codes and regulations guide researchers and sponsors. Review boards and peer groups 
help researchers examine their research proposals for ethical dilemmas. Many design-based ethical 
problems can be eliminated by careful planning and constant vigilance. In the end, responsible research 
anticipates ethical dilemmas and attempts to adjust the design, procedures, and protocols during the 
planning process rather than treating them as an afterthought. Ethical research requires personal integ- 
rity from the researcher, the project manager, and the research sponsor. 

Because integrity in research is vital, we are discussing its components early in this book and em- 
phasizing ethical behavior throughout our coverage. Our objective is to stimulate an ongoing exchange 
about values and practical research constraints in the chapters that follow. This chapter is organized 
around the theme of ethical treatment of participants, clients, research sponsors, and other research- 
ers. We also highlight appropriate laws and codes, resources for ethical awareness, and cases for ap- 
plication. Exhibit 2-1 relates each ethical issue under discussion to the research process introduced in 
Chapter 1. 


> Ethical Treatment of Participants 


When ethics are discussed in research design, we often think first about protecting the rights of the par- 
ticipant, or subject. Whether data are gathered in an experiment, interview, observation, or survey, the 
participant has many rights to be safeguarded. In general, research must be designed so that a partici- 
pant does not suffer physical harm, discomfort, pain, embarrassment, or loss of privacy. To safeguard 
against these, the researcher should follow three guidelines:* 


1. Explain study benefits. 
2. Explain participant rights and protections. 


3. Obtain informed consent. 
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>Exhibit 2-1 Ethical Issues and the Research Process 


~<#——- Sponsor nondisclosure 


Exploration Exploration 


e Sponsor’s right to quality 
research 

¢ Sponsor’s right of purpose 
nondisclosure 

e Researcher’s right to absence 
of sponsor coercion 

e Researcher's right to absence 
of sponsor deception 


Research Design 
Strategy 


e Participant deception 
(type, purpose, time frame, scope, environment) 


¢ Sponsor’s right to quality 
research 


Data Collection Sampling 


| e Participant’s right of informed 
Design 


consent 

e Participant’s right to privacy 
(refusal) 

¢ Sponsor’s right to quality 
research 

e Researcher’s right to absence 
of sponsor coercion 


¢ Sponsor’s right to quality 
research 

e Participant’s right to privacy 

e Participant deception 

e Sponsor’s right to sponsor 
nondisclosure 

e Researcher’s right to safety 


¢ Sponsor’s right to findings 
nondisclosure 

3 e Participant’s right to 

Research Reporting confidentiality 

¢ Sponsor’s right to quality 
research 

e Researcher’s right to absence 
of sponsor coercion 


Management 
Decision 


Benefits 


Whenever direct contact is made with a participant, the researcher should discuss the study’s ben- 
efits, being careful to neither overstate nor understate the benefits. An interviewer should begin an 
introduction with his or her name, the name of the research organization, and a brief description of 
the purpose and benefit of the research. This puts participants at ease, lets them know to whom they 
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Privacy’s New Frontier . . . Location-Based Services 


With the continued growth in use of smart phones, the ability 
to run software and mobile applications using wireless location- 
based services (LBS) is a major attraction for both businesses 
and customers. With LBS, individuals share real-time and histori- 
cal location information online to facilitate a variety of activities, 
including find friends, play games, locate a restaurant, or shop 
for a bargain. Facebook Places, Living Social, Groupon, and 
ShopAlert are just a few of the current players. 

“But alongside the benefits, mobile LBS capabilities also in- 
volve consumer privacy risks. For example, the top perceived 
consumer risk of LBS-enabled services is the unintended revela- 
tion of a user’s home address. Websites like ‘Please Rob Me’ 
demonstrate the danger of location-sharing by providing a data- 
base of empty homes based on users’ ‘check-ins’ elsewhere,” 
shared Alysa Zeltzer Hutnik, partner in the privacy and security 
practice of Kelley Drye & Warren LLP and communications man- 
ager, International Association of Privacy Professionals. 

While consumers have expressed concerns about busi- 
nesses tracking their Web behavior to target ads, a majority 
(62%) expressed a “willingness to let their usage patterns and 
personal information be tracked by advertisers if this resulted 
in lower product costs or free online content, and 43 percent of 
consumers said they were willing to receive targeted advertising 
in exchange for lower fees or service.” 

Carnegie Mellon University studied the privacy controls of 89 
popular LBS applications in 2010. Of those surveyed, 66 per- 
cent had some form of privacy policy. And most of these re- 
tained all information for an indefinite period of time. By 2011, a 
TrustE study revealed “97 percent of the top 100 sites had some 


form of privacy policy in place, though many companies had a 
weak understanding of their privacy policies and the third-party 
software used on their website.” 

“Even in the absence of black-letter law on LBS practices, 
“shared Hutnik,” there are some clear dos and don’ts that are 
worth considering when engaging in business practices involving 
LBS-enabled services.” 


e Businesses should know what their LBS service does, 
what type of data it collects, and whether that data is 
shared with affiliates, partners, or third parties. 

e Businesses should appoint privacy-trained personnel 
to ensure that privacy considerations are identified 
appropriately and satisfied, both at the outset of the 
design of a new service or product, as well as at periodic 
intervals after the service or product is released publicly. 

e Businesses should treat LBS information collection 
and disclosure as sensitive personal information, which 
means being transparent and careful with the data. 

e Businesses should demonstrate that they have obtained 
informed consent to the use or disclose location 
information before initiating an LBS service. 

e Businesses should be sensitive to parent expectations 
as well as the additional legal scrutiny that follows 
any marketing efforts targeted to young people under 
13 years of age. 

e Businesses should stay current on privacy developments 
and resources, including activities of the FTC and states’ 
attorney general. 


are speaking, and motivates them to answer questions truthfully. In short, knowing why one is being 
asked questions improves cooperation through honest disclosure of purpose. Inducements to partici- 
pate, financial or otherwise, should not be disproportionate to the task or presented in a fashion that 
results in coercion. 

Sometimes the actual purpose and benefits of your study or experiment must be concealed from the 
participants to avoid introducing bias. The need for concealing objectives leads directly to the problem 
of deception. 


Deception 


Deception occurs when the participants are told only part of the truth or when the truth is fully 
compromised. Some believe this should never occur. Others suggest two reasons for deception: (1) 
to prevent biasing the participants before the survey or experiment and (2) to protect the confiden- 
tiality of a third party (e.g., the sponsor). Deception should not be used in an attempt to improve 
response rates. 

The benefits to be gained by deception should be balanced against the risks to the participants. When 
possible, an experiment or interview should be redesigned to reduce reliance on deception. In addition, 
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the participants’ rights and well-being must be adequately protected. In instances where deception in 
an experiment could produce anxiety, a subject’s medical condition should be checked to ensure that 
no adverse physical harm follows. The American Psychological Association’s ethics code states that 
the use of deception is inappropriate unless deceptive techniques are justified by the study’s expected 
scientific, educational, or applied value and equally effective alternatives that do not use deception are 
not feasible.* And, finally, the participants must have given their informed consent before participating 
in the research. 


Informed Consent 


Securing informed consent from participants is a matter of fully disclosing the procedures of the pro- 
posed survey or other research design before requesting permission to proceed with the study. There 
are exceptions that argue for a signed consent form. When dealing with children, the researcher is wise 
to have a parent or other person with legal standing sign a consent form. When doing research with 
medical or psychological ramifications, the researcher is also wise to obtain a signed consent form. If 
there is a chance the data could harm the participant or if the researchers offer only limited protection 
of confidentiality, a signed form detailing the types of limits should be obtained. For most business 
research, oral consent is sufficient. An example of how informed-consent procedures are implemented 
is shown in Exhibit 2-2. In this example, a university research center demonstrates how it adheres to 
the highest ethical standards for survey procedures.* 

In situations where participants are intentionally or accidentally deceived, they should be debriefed 
once the research is complete. 


>Exhibit 2-2 |Informed-Consent Procedures for Surveys 


Surveys conducted by the Indiana University Center for Survey Research contain the following informed-consent components in their 


introductions: 


= 


. Introduce ourselves—interviewer’s name and Indiana University Center for Survey Research. 


. Briefly describe the survey topic (e.g., barriers to health insurance). 


. Tell who the sponsor is (e.g., National Endowment for the Humanities). 

. Describe the purpose(s) of the research (e.g., satisfaction with services received/provided by a local agency). 
. Give a “good-faith” estimate of the time required to complete the interview. 

. Promise anonymity and confidentiality (when appropriate). 

. Tell the participant the participation is voluntary. 


. Tell the participant that item-nonresponse is acceptable. 


(Ss) Gey fo) SS) Se) 


ar 


. Ask permission to begin. 


Hello, !’m fill in NAME] from the Center for Survey Research at Indiana University. We’re surveying Indianapolis area residents to ask 
their opinions about some health issues. This study is sponsored by the National Institutes of Health and its results will be used to 


. Describe the geographic area we are interviewing (e.g., people in Indiana) or target sample (e.g., aerospace engineers). 


research the effect of community ties on attitudes toward medical practices. The survey takes about 40 minutes. Your participation is 
anonymous and voluntary, and all your answers will be kept completely confidential. If there are any questions that you don’t feel you 


can answer, please let me know and we’ll move to the next one. So, if | have your permission, |’ll continue. 


The participant is given information on how to contact the principal investigator. For example: John Kennedy is the principal investigator for 


this study. Would you like Dr. Kennedy’s address or telephone number in case you want to contact him about the study at any time? 
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Researchers have special ethical 
responsibilities when using children 
as participants. Quality researchers 
are fully compliant with the Children’s 
Online Privacy Protection Act 
(COPPA). COPPA federal regulations 
ensure that proper parental 
permission is obtained prior to 
initiating contact. Besides providing 
informed consent, parents are often 
interviewed during the selection 
process to ensure that if their child is 
chosen, he or she is mature enough 
to handle the activities planned 

and has the verbal and physical 
capabilities necessary. Researchers 
who work with children want the 
child to perceive participation as an 
enjoyable—and sometimes even 

an exciting — experience. 
www.coppa.org 


Debriefing Participants 


Debriefing involves several activities following the collection of data: 


e Explanation of any deception. 
¢ Description of the hypothesis, goal, or purpose of the study. 
e Poststudy sharing of results. 


¢ Poststudy follow-up medical or psychological attention. 


First, the researcher shares the truth of any deception with the participants and the reasons for using deception 
in the context of the study’s goals. In cases where severe reactions occur, follow-up medical or psychological 
attention should be provided to continue to ensure the participants remain unharmed by the research. 

Even when research does not deceive the participants, it is a good practice to offer them follow-up infor- 
mation. This retains the goodwill of the participant, providing an incentive to participate in future research 
projects. For surveys and interviews, participants can be offered a brief report of the findings. Usually, they 
will not request additional information. Occasionally, however, the research will be of particular interest 
to a participant. A simple set of descriptive charts or data tables can be generated for such an individual. 

For experiments, all participants should be debriefed in order to put the experiment into context. 
Debriefing usually includes a description of the hypothesis being tested and the purpose of the study. 
Participants who were not deceived still benefit from the debriefing session. They will be able to un- 
derstand why the experiment was created. The researchers also gain important insight into what the 
participants thought about during and after the experiment. This may lead to modifications in future 
research designs. Like survey and interview participants, participants in experiments and observational 
studies should be offered a report of the findings. 

To what extent do debriefing and informed consent reduce the effects of deception? Research sug- 
gests that the majority of participants do not resent temporary deception and may have more positive 
feelings about the value of the research after debriefing than those who didn’t participate in the study.° 
Nevertheless, deception is an ethically thorny issue and should be addressed with sensitivity and con- 
cern for research participants. 


Rights to Privacy 


Privacy laws in the United States are taken seriously. All individuals have a right to privacy, and re- 
searchers must respect that right. The importance of the right to privacy is illustrated with an example. 
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ESOMAR and CASRO on the Ethics of Mobile Surveys 


ESOMAR is a world organization focused on enabling better 
research into markets, consumers, and societies. It facilitates 
an ongoing dialogue with its 4,900 members, in more than 130 
countries, to promote the value of market and opinion research 
in effective decision making. Council of American Survey Re- 
search Organizations (CASRO) represents more than 300 com- 
panies and research operations in the United States and abroad. 
It considers itself the “voice and values” of the survey research 
industry. Both organizations have been on the forefront of de- 
veloping guidelines for mobile research practices. While many 
guidelines relate to surveys regardless of the mode of collection, 
mobile surveys have some unique guidelines. 


Recruiting Participants 

While most countries do not restrict unsolicited calls for research, 
ESOMAR guidelines indicate it is “mandatory to consult and 
apply [country-specific], research-specific do-not-contact lists” 
for mobile phones, if such rules exist. Prior consent is required 
for unsolicited text messages in a number of countries although 
these regulations may not always apply to research. If text mes- 
saging is used to contact respondents, ESOMAR guidelines 
require that “the researcher shall provide the opportunity to opt 
out in order to meet [the requirement of prior consent] to ensure 
the voluntary nature of research participation. The CASRO code 
additionally requires that survey research organizations will not 
use unsolicited emails to recruit survey respondents. 


Preventing Financial Disadvantage 

Calls to mobile devices can involve a charge to both the caller 
and the recipient. And roaming charges may be incurred when 
survey calls are made to mobile numbers across regional and 
national boundaries. As a result, a researcher may be in breach 
of their obligation to ensure that a potential respondent is in “no 
way disadvantaged by participation in a research survey.” 


Privacy and Intrusion 

ESOMAR guidelines require that a researcher “politely termi- 
nate the call when it becomes apparent that the recipient is not 
in a position (e.g., driving a car, operating machinery, walking 


in a public space, or when the caller is in another country/ 
time zone) or does not wish to take the call, is not competent 
(to answer the survey), or is a child (unless the researcher re- 
ceives verifiable, informed consent from an appropriate adult 
to proceed with the call).” Thus ESOMAR guidelines require a 
researcher to take all reasonable precautions to ensure that 
respondents are not harmed or adversely affected as a direct 
result of participating in an interview or survey. Researchers 
are encouraged to confirm whether the potential respondent 
is in a situation where it is legal, safe, and convenient to take 
the call. 

Since a respondent on a mobile device could be reached in 
a public or semi-private space, the ESOMAR guidelines require 
the researcher to “consider the nature of the survey content in 
light of the possibility that the respondent might be overheard 
and personal information or behavior inadvertently disclosed or 
responses modified due to the respondent’s situation.” 


Mobile Device Alteration to Track Behavior 

Mobile surveys require active agent technology, defined as “any 
software or hardware device that captures the behavioral data 
about subjects in a background mode, typically running concur- 
rently with other activities.” This technology includes direct to 
device software downloaded to a user’s device that is “used 
solely for the purpose of alerting potential survey respondents, 
downloading survey content or asking survey questions.” Trans- 
parency is the primary guideline for mobile research, requiring 
that the researcher provide the potential participant with suf- 
ficient information to opt in or opt out of a survey—including 
what the software tracks, how long the software resides on the 
participant’s device or guidelines for its removal, that it not con- 
tain advertising (unless it is the subject of the research), and 
that prior consent is obtained before software installation. Many 
mobile devices are enabled with geo-spatial devices, capable of 
physically locating the participant. Guidelines require that such 
information requires prior consent and, if collected, be kept 
confidential. 


Wwww.coppa.org; www.esomar.org 


An employee of MonsterVideo, a large video company, is also a student at the local university. For 
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a research project, this student and his team members decide to compare the video-viewing habits of a 
sample of customers. Using telephone interviews, the students begin their research. After inquiring about 
people’s viewing habits and the frequency of rentals versus purchases, the students move on to the types 
of films people watch. They find that most participants answer questions about their preferences for chil- 
dren’s shows, classics, best-sellers, mysteries, and science fiction. But the cooperation ceases when the 
students question the viewing frequency of pornographic movies. Without the guarantee of privacy, most 
people will not answer these kinds of questions truthfully, if at all. The study then loses key data. 
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Has Trust Trumped Privacy? 


Since e-commerce rang up its first sale, privacy advocates have 
been telling business that privacy is an important issue among 
online browsers and purchasers. An early Harris Poll survey con- 
ducted for BusinessWeek showed more than one-third of U.S. 
adults would be uncomfortable with their online actions being pro- 
filed, while 82 percent would be uncomfortable with online activi- 
ties being merged with personally identifiable information such as 
income, driver’s license number, credit data, and medical status. 
The most recent TrustE Privacy Index showed “91 percent of U.S. 
online adults worry to some degree about their privacy online, while 
53 percent said they ‘don’t completely trust companies with their 
business online’.” This continues to reflect work by researcher. 
Alan Westin, president and publisher of Privacy & American Busi- 
ness, who identified three different groups related to privacy: pri- 


further erosion), privacy pragmatists (who are willing to share per- 
sonal information when they understand the reasons for its use 
or see tangible benefits from sharing), and privacy unconcerned 
(those for whom privacy is not an overriding concern). 

Even in the face of the inappropriate release of large private 
information data bases by government, financial institutions, and 
retailers entrusted with the information, this most recent study 
by Truste shows a narrowing of adults’ privacy concerns. Eighty- 
eight percent of U.S. adults avoid doing business with firms that 
don’t protect their personal information and 93 percent believe 
businesses have a responsibility to protect their privacy. That’s 
why you saw a rash of businesses updating and simplifying their 
privacy statements in 2012. 


www.truste.org; www.harrisinteractive.com 


vacy fundamentalists (who feel they have lost their privacy and fear 


The privacy guarantee is important not only to retain validity of the research but also to protect 
participants. In the previous example, imagine the harm that could be caused by releasing information 
on the viewing habits of certain citizens. Clearly, the confidentiality of survey answers is an important 
aspect of the participants’ right to privacy. Once the guarantee of confidentiality is given, protecting 
that confidentiality is essential. The researcher protects participant confidentiality in several ways: 


° Obtaining signed nondisclosure documents. 

° Restricting access to participant identification. 

¢ Revealing participant information only with written consent. 

e Restricting access to data instruments where the participant is identified. 
e Not disclosing data subsets. 


Researchers should restrict access to information that reveals names, telephone numbers, addresses, 
or other identifying features. Only researchers who have signed nondisclosure, confidentiality forms 
should be allowed access to the data. Links between the data or database and the identifying information 
file should be weakened. Individual interview response sheets should be inaccessible to everyone ex- 
cept the editors and data entry personnel. Occasionally, data collection instruments should be destroyed 
once the data are in a data file. Data files that make it easy to reconstruct the profiles or identification of 
individual participants should be carefully controlled. For very small groups, data should not be made 
available because it is often easy to pinpoint a person within the group. Employee-satisfaction survey 
feedback in small units can be easily used to identify an individual through descriptive statistics alone. 
These last two protections are particularly important in human resources research.° 

But privacy is more than confidentiality. A right to privacy means one has the right to refuse to be 
interviewed or to refuse to answer any question in an interview. Potential participants have a right to 
privacy in their own homes, including not admitting researchers and not answering telephones. And 
they have the right to engage in private behavior in private places without fear of observation. To ad- 
dress these rights, ethical researchers do the following: 


° Inform participants of their right to refuse to answer any questions or participate in the study. 
¢ Obtain permission to interview participants. 

e Schedule field and phone interviews. 

e Limit the time required for participation. 

¢ Restrict observation to public behavior only. 
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Data Collection in Cyberspace 


Some ethicists argue that the very conduct that results in resistance from participants—interference, 
invasiveness in their lives, denial of privacy rights—has encouraged researchers to investigate topics 
online that have long been the principal commodity of offline investigation. The novelty and conve- 
nience of communicating by computer has led researchers to cyberspace in search of abundant sources 
of data. Whether we call it the “wired society,” “digital life,” “computer-mediated communication,” or 
“cyberculture,” the growth of cyberstudies causes us to question how we gather data online, deal with 
participants, and present results. 

In a special ethics issue of Information Society, scholars involved in cyberspace research concluded: 


All participants agree that research in cyberspace provides no special dispensation to ignore ethical precepts. Researchers 
are obligated to protect human subjects and “do right” in electronic venues as in more conventional ones. Second, each 
participant recognizes that cyberspace poses complex ethical issues that may lack exact analogs in other types of inquiry. 
The ease of covert observation, the occasional blurry distinction between public and private venues, and the difficulty of ob- 
taining the informed consent of subjects make cyber-research particularly vulnerable to ethical breaches by even the most 
scrupulous scholars. Third, all recognize that because research procedures or activities may be permissible or not pre- 
cluded by law or policy, it does not follow that they are necessarily ethical or allowable. Fourth, all agree that the individual 
researcher has the ultimate responsibility for assuring that inquiry is not only done honestly, but done with ethical integrity.” 


Issues relating to cyberspace in research also relate to data mining. The information collection 
devices available today were once the tools of the spy, the science fiction protagonist, or the super- 
hero. Smart cards, biometrics (finger printing, retinal scans, facial recognition), electronic monitoring 
(closed circuit television, digital camera monitoring), global surveillance, and genetic identification 
(DNA) are just some of the technological tools being used by today’s organizations to track and un- 
derstand employees, customers, and suppliers. The data mining of all this information, collected from 
advanced and not necessarily obvious sources, offers infinite possibilities for research abuse. 


Data Mining Ethics 


The primary ethical data-mining issues in cyberspace are privacy and consent. (See Exhibit 2-3.) Smart 
cards, those ubiquitous credit card—sized devices that embed personal information on a computer chip that 


>Exhibit 2-3 The Seven Basic Principles of the U.S. Safe Harbor Agreement 
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¢ Notice. Companies must notify consumers/participants about what information is being collected, how that information will be 
used, whom that information will be shared with, and how individuals can contact the organization with any inquiries or complaints. 


* Choice. Consumers/participants must be provided with an opt-out mechanism for any secondary uses of data and for disclosures 
to third parties. For sensitive information, participants must opt in before providing data that will be shared. 


¢ Access. Individuals must have access to personal information about themselves that an organization holds and be able to cor- 
rect, amend, or delete that information where it is inaccurate, except where the burden or expense of providing access would be 
disproportionate to the risks to the individual’s privacy. 


Security. Organizations must take reasonable precautions to protect personal information from loss, misuse, and unauthorized 
access, disclosure, alteration, and destruction. 


* Onward transfer. Companies disclosing personal data to a third party must, with certain exceptions, adhere to the notice and 
choice principles. A third party must subscribe to the safe-harbor principles. 


Data integrity. Reasonable steps must be taken to ensure that data collected are reliable, accurate, complete, and current. 


¢ Enforcement. Companies must ensure there are readily available and affordable independent mechanisms to investigate consumer 
complaints, obligations to remedy problems, procedures to verify compliance with safe-harbor principles, and sufficiently rigorous 
sanctions to ensure compliance. 


Source: Diane Bowers, “Privacy and the Research Industry in the U.S.,” ESOMAR Research World, no. 7, July-August 2001, pp. 8-9 


(http://www.esomar.nl/PDF/DataPrivacyUpdateUSA.pdf); Lori Enos, “Microsoft to Sign EU Privacy Accord,” www.EcommerceTimes.com, 


May 16, 2001 (http://www.newsfactor.com/perl/story/9752.html); U.S. Department of Commerce, “Safe Harbor Overview,” accessed 
November 30, 2002 (http://www.export.gov/safeharbor/sh_overview.html). 
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is then matched to purchase, employment, or other behavior data, offer the researcher implied consent to 
participant surveillance. But the benefits of card use may be enough to hide from an unsuspecting user the 
data-mining purpose of the card. For example, The Kroger Co., one of the largest grocers in the United 
States, offers significant discounts for enrollment in its Kroger Plus Shopper’s Card program.® Retailers, 
wholesalers, medical and legal service providers, schools, government agencies, and resorts, to name 
a few, use smart cards or their equivalent. In most instances, participants provide, although sometimes 
grudgingly, the personal information requested by enrollment procedures. But in others, enrollment is 
mandatory, such as when smart cards are used with those convicted of crimes and sentenced to municipal 
or state correction facilities or those attending specific schools. In some instances, mandatory sharing of 
information is initially for personal welfare and safety—such as when you admit yourself for a medical 
procedure and provide detailed information about medication or prior surgery. But in others, enrollment 
is for less critical but potentially attractive monetary benefits—for example, free car care services when a 
smart card is included with the keys to a new vehicle. The bottom line is that the organization collecting 
the information gains a major benefit: the potential for better understanding and competitive advantage. 


European Union 


General privacy laws may not be sufficient to protect the unsuspecting in the cyberspace realm of data 
collection. The 15 European Union (EU) countries started the new century by passing the European 
Commission’s Data Protection Directive. Under the directive, commissioners can prosecute companies 
and block websites that fail to live up to its strict privacy standards. Specifically, the directive prohibits 
the transmission of names, addresses, ethnicity, and other personal information to any country that fails 
to provide adequate data protection. This includes direct mail lists, hotel and travel reservations, medi- 
cal and work records, and orders for products, among a host of others.° U.S. industry and government 
agencies have resisted regulation of data flow. But the EU insists that it is the right of all citizens to find 
out what information about themselves is in a database and correct any mistakes. Few U.S. companies 
would willingly offer such access due to the high cost;'° a perfect example of this reluctance is the dif- 
ficulty individuals have correcting erroneous credit reports, even when such information is based on 
stolen personal identity or credit card transactions. 

Yet questions remain regarding the definition of specific ethical behaviors for cyber-research, the 
sufficiency of existing professional guidelines, and the issue of ultimate responsibility for participants. 
If researchers are responsible for the ethical conduct of their research, are they solely responsible for 
the burden of protecting participants from every conceivable harm? 


> Ethics and the Sponsor 


There are also ethical considerations to keep in mind when dealing with the research client or sponsor. 
Whether undertaking product, market, personnel, financial, or other research, a sponsor has the right to 
receive ethically conducted research. 


Confidentiality 


Some sponsors wish to undertake research without revealing themselves. They have a right to sev- 
eral types of confidentiality, including sponsor nondisclosure, purpose nondisclosure, and findings 
nondisclosure. 

Companies have a right to dissociate themselves from the sponsorship of a research project. This type of 
confidentiality is called sponsor nondisclosure. Due to the sensitive nature of the management dilemma or 
the research question, sponsors may hire an outside consulting or research firm to complete research projects. 
This is often done when a company is testing a new product idea, to prevent potential consumers from being 
influenced by the company’s current image or industry standing, or if a company is contemplating entering 
anew market, it may not wish to reveal its plans to competitors. In such cases, it is the responsibility of the 
researcher to respect this desire and devise a plan that safeguards the identity of the research sponsor. 
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groups. 


Purpose nondisclosure involves protecting the purpose of the study or its details. A research sponsor 
may be testing a new idea that is not yet patented and may not want the competition to know of its plans. 
It may be investigating employee complaints and may not want to spark union activity, or the sponsor 
might be contemplating a new public stock offering, in which advance disclosure would spark the interest 
of authorities or cost the firm thousands or millions of dollars. Finally, even if a sponsor feels no need to 
hide its identity or the study’s purpose, most sponsors want the research data and findings to be confiden- 
tial, at least until the management decision is made. Thus sponsors usually demand and receive findings 
nondisclosure between themselves or their researchers and any interested but unapproved parties. 


The Sponsor-Researcher Relationship 


In an organizational setting, the researcher should look on the sponsoring manager as a client. An ef- 
fective sponsor-—researcher relationship is not achieved unless both fulfill their respective obligations 
and several critical barriers are overcome. 

The obligations of managers are to specify their problems and provide researchers with adequate 
background information and access to company information gatekeepers. It is usually more effective if 
managers state their problems in terms of the decision choices they must make rather than the information 
they want. If this is done, both manager and researcher can jointly decide what information is needed. 

Researchers also have obligations. Organizations expect them to develop a creative research design 
that will provide answers to important business questions. Not only should researchers provide data 
analyzed in terms of the problem specified, but they also should point out the limitations that affect 
the results. In the process, conflict may arise between what the decision maker wants and what the 
researcher can provide ethically or thinks should be provided. The sponsor wants certainty and simple, 
explicit recommendations, while the researcher often can offer only probabilities and hedged inter- 
pretations. This conflict is inherent in their respective roles and has no simple resolution. However, a 
workable balance can usually be found if each person is sensitive to the demands, ethical constraints, 
and restrictions imposed on the other. 

Among the sources of manager-researcher conflict are: 


° Knowledge gap between the researcher and the manager. 
e Job status and internal, political coalitions to preserve status. 
¢ Unneeded or inappropriate research. 


¢ The right to quality research. 


Information can make or break a 
business in the rapidly changing 
world of business. That’s why you 
need a researcher that can extract 
information while keeping results 
strictly confidential. Seaport Surveys 
is one such firm. It specializes in 
recruiting hard-to-reach respondents, 
as well as business-to-business 
interviewing and executive focus 


www.seaportsurveys.com 
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Knowledge Gap 


Some conflicts between decision makers and researchers are traced to management’s limited exposure 
to research. Managers seldom have either formal training in research methodology, the various aspects 
of research ethics, or research expertise gained through experience. And, due to the explosive growth 
of research technology in recent years, a knowledge gap has developed between managers and research 
specialists as more sophisticated investigative techniques have come into use. Thus, the research spe- 
cialist removes the manager from his or her comfort zone: the manager must now put his or her faith, 
and sometimes his or her career, in the hands of the research specialist’s knowledge and adherence to 
ethical standards. 


Job Status and Internal Coalitions 


In addition, managers often see research people as threats to their personal status. Managers still view 
management as the domain of the “intuitive artist’ who is the master in this area. They may believe 
a request for research assistance implies they are inadequate to the task. These fears are often justi- 
fied. The researcher’s function is to test old ideas as well as new ones. To the insecure manager, the 
researcher is a potential rival. 

The researcher will inevitably have to consider the corporate culture and political situations that 
develop in any organization. Members strive to maintain their niches and may seek ascendancy 
over their colleagues. Coalitions form and people engage in various self-serving activities, both 
overt and covert. As a result, research is blocked, or the findings or objectives of the research are 
distorted for an individual’s self-serving purposes. To allow one’s operations to be probed with a 
critical eye may be to invite trouble from others competing for promotion, resources, or other forms 
of organizational power. 


Unneeded or Inappropriate Research 


Not every managerial decision requires research. Business research has an inherent value only to the 
extent that it helps management make better decisions. Interesting information about consumers, em- 
ployees, or competitors might be pleasant to have—but its value is limited if the information cannot be 
applied to a critical decision. If a study does not help management select more efficient, less risky, or 
more profitable alternatives than otherwise would be the case, the researcher has an ethical responsibil- 
ity to question its use. 


Right to Quality Research 


An important ethical consideration for the researcher and the sponsor is the sponsor’s right to quality 
research. This right entails: 


e Providing a research design appropriate for the research question. 
e Maximizing the sponsor’s value for the resources expended. 


e Providing data-handling and data-reporting techniques appropriate for the data collected. 


From the proposal through the design to data analysis and final reporting, the researcher guides 
the sponsor on the proper techniques and interpretations. Often sponsors will have heard about 
a sophisticated data-handling technique and will want it used even when it is inappropriate for 
the problem at hand. The researcher should guide the sponsor so that this does not occur. The 
researcher should propose the design most suitable for the problem. The researcher should not 
propose activities designed to maximize researcher revenue or minimize researcher effort at the 
sponsor’s expense. 

Finally, we have all heard the remark, “You can lie with statistics.” It is the researcher’s respon- 
sibility to prevent that from occurring. The ethical researcher always follows the analytical rules and 
conditions for results to be correct. The ethical researcher reports findings in ways that minimize the 
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drawing of false conclusions. The ethical researcher also uses charts, graphs, and tables to show the 
data objectively, despite the sponsor’s preferred outcomes. 


Sponsor’s Ethics 


Occasionally, research specialists may be asked by sponsors to participate in unethical behavior. 
Compliance by the researcher would be a breach of ethical standards. Some examples to be avoided 
are: 


¢ Violating participant confidentiality. 

e Changing data or creating false data to meet a desired objective. 
¢ Changing data presentations or interpretations. 

¢ Interpreting data from a biased perspective. 

e Omitting sections of data analysis and conclusions. 


e Making recommendations beyond the scope of the data collected. 


Let’s examine the effects of complying with these types of coercion. A sponsor may offer a 
promotion, future contracts, or a larger payment for the existing research contract; or the sponsor may 
threaten to fire the researcher or tarnish the researcher’s reputation. For some researchers, the request 
may seem trivial and the reward high. But imagine for a moment what will happen to the researcher 
who changes research results. Although there is a promise of future research, can the sponsor ever trust 
that researcher again? If the researcher’s ethical standards are for sale, which sponsor might be the 
highest bidder next time? Although the promise of future contracts seems enticing, it is unlikely to be 
kept. Each coercive reward or punishment has an equally poor outcome. The “greater than” contracted 
payment is a payoff. The threats to one’s professional reputation cannot be carried out effectively by a 
sponsor who has tried to purchase you. So the rewards for behaving unethically are illusory. 

What’s the ethical course? Often, it requires confronting the sponsor’s demand and taking the fol- 
lowing actions: 


e Educate the sponsor to the purpose of research. 
e Explain the researcher’s role in fact finding versus the sponsor’s role in decision making. 
e Explain how distorting the truth or breaking faith with participants leads to future problems. 


e Failing moral suasion, terminate the relationship with the sponsor. 


> Researchers and Team Members 


Another ethical responsibility of researchers is their team’s safety as well as their own. In addition, the 
responsibility for ethical behavior rests with the researcher who, along with assistants, is charged with 
protecting the anonymity of both the sponsor and the participant. 


Safety 


It is the researcher’s responsibility to design a project so that the safety of all interviewers, surveyors, 
experimenters, or observers is protected. Several factors may be important to consider in ensuring a 
researcher’s right to safety. Some urban areas and undeveloped rural areas may be unsafe for research 
assistants. If, for example, the researcher must personally interview people in a high-crime district, it 
is reasonable to provide a second team member to protect the researcher. Alternatively, if an assistant 
feels unsafe after visiting a neighborhood by car, an alternate researcher should be assigned to the des- 
tination.'' It is unethical to require staff members to enter an environment where they feel physically 
threatened. Researchers who are insensitive to these concerns face both research and legal risks—the 
least of which involves having interviewers falsify instruments. 
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One consequence of the 

financial crisis was multiple 

home foreclosures. In some 
neighborhoods, these abandoned 
properties have changed the very 
landscape of safety for residents 
and visitors alike. Methodologies 
that would require researchers to 
visit such neighborhoods would 
need to be carefully assessed. If 
a visit was deemed crucial to the 
research, significant safeguards 
would need to be in place. 
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Ethical Behavior of Assistants 


Researchers should require ethical compliance from team members just as sponsors expect ethical behav- 
ior from the researcher. Assistants are expected to carry out the sampling plan, to interview or observe 
participants without bias, and to accurately record all necessary data. Unethical behavior, such as filling 
in an interview sheet without having asked the participant the questions, cannot be tolerated. The behavior 
of the assistants is under the direct control of the responsible researcher or field supervisor. If an assistant 
behaves improperly in an interview or shares a participant’s interview sheet with an unauthorized person, 
it is the researcher’s responsibility. Consequently, all assistants should be well trained and supervised. 


Protection of Anonymity 


As discussed previously, researchers and assistants protect the confidentiality of the sponsor’s informa- 
tion and the anonymity of the participants. Each researcher handling data should be required to sign a 
confidentiality and nondisclosure statement. 


> Professional Standards 


Various standards of ethics exist for the professional researcher. Many corporations, professional as- 
sociations, and universities have a code of ethics. The impetus for these policies and standards can be 
traced to two documents: the Belmont Report of 1979 and the Federal Register of 1991." Society or 
association guidelines include ethical standards for the conduct of research. One comprehensive source 
contains 51 official codes of ethics issued by 45 associations in business, health, and law.'* The busi- 
ness section of this source consists of ethics standards for: 


Accounting—American Institute of Certified Public Accountants. 
Advertising—American Association of Advertising Agencies; Direct Marketing Association. 
Banking—American Bankers Association. 


Engineering—American Association of Engineering Societies; National Society of Professional 
Engineers. 


Financial planning—Association for Investment Management and Research; Certified Financial 


>chapter 2 Ethics in Business Research 


Is Your Research Project Leaving the Country? 


Offshoring is defined as the movement of a process done at a 
company in one country to the same or another company in 
a different country. These processes in research often include 
IT (information technology), business, or knowledge processes. 
The primary reason to offshore research services is to lower the 
cost of the research. Offshoring tends to be more prevalent at 
larger research firms. 

There are significant risks associated with offshoring re- 
search services, as Gordon Morris, global insights manager 
for Sony Ericsson (London, UK) discovered. During a global 
research project for the Experia X10 Android phone, sensitive 
business plans were shared with the contracted researchers. 
As a result of offshoring of some research services, information 
about the 2010 phone launch leaked several months early. “We 
estimate the potential damage caused by the leak at approxi- 
mately £100 million,” shared Morris. Leaks may be more likely 
to occur when offshoring is used because intellectual property 
standards, safe computing standards, and 
contract laws vary from country to country. 


Tom H. C. Anderson, FTO founder and chairman, and managing 
partner of the research consultancy Anderson Analytics. While 
not taking a stand for or against offshoring, the FTO encourages 
research companies to register their project practices and earn 
one of two seals: one seal certifies that the research organiza- 
tions that do offshore research services comply with the FTO 
disclosure standards, which are modeled after the EU’s Safe 
Harbour Compliance Framework; the second seal identifies re- 
search organizations that do not offshore services at all. 

FTO hopes that research buyers will look for the certifica- 
tion seals when purchasing research services. As Sonia Baldia, 
partner at Mayer Brown LLP and a legal expert on offshoring 
explained, “Clients absolutely need to know about any offshore 
subcontracting and the location in order to gauge risks and pro- 
tect themselves.” 


www.offshoringtransparency.org 


Q. Regardless of what your stance is on offshoring, do you believe research 


suppliers have an obligation to tell their clients that they offshore? 


Also, high employee turnover in some de- 
veloping countries can also add to this risk. 
In 2009 a new trade group was formed 90 


100% - 


to encourage transparency in offshoring g0 L 
of research services: the Foundation for 

Transparency in Offshoring (FTO). A sur- eal 
vey fielded to 850 U.S. and international 60 - 
research buyers and providers indicated 50 - 
that clients were more likely to think their 40 - 
projects did not involve offshoring than 30 b 
was actually true. Clients also were much 

more likely than research services suppli- saul 
ers to think clients should be told about 10 - 

0 | L | 


™ Yes 
® No 


offshoring. “Very few buyers have sufficient 
information to assess the relative strengths 
and risks associated with offshoring,” said 


Suppliers 


Non- Offshoring Clients 
offshoring Supplier 
Supplier 


Planner Board of Standards/Institute of Certified Financial Planners; International Association for 


Financial Planning. 


Human resources—American Society for Public Administration; Society for Human Resource 


Management. 


Insurance—American Institute for Chartered Property Casualty Underwriters; American Society 
of Chartered Life Underwriters and Chartered Financial Consultants. 


Management—Academy of Management; The Business Roundtable. 


Real estate—National Association of Realtors. 


Other professional associations’ codes have detailed research sections: the American Marketing 
Association, the American Association for Public Opinion Research, the American Psychological 
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Association, the American Political Science Association, the American Sociological Association, and 
the Society of Competitive Intelligence Professionals. These associations update their codes frequently. 

We commend professional societies and business organizations for developing standards. However, 
without enforcement, standards are ineffectual. Effective codes (1) are regulative, (2) protect the public 
interest and the interests of the profession served by the code, (3) are behavior-specific, and (4) are 
enforceable. A study that assessed the effects of personal and professional values on ethical consulting 
behavior concluded: 


The findings of this study cast some doubt on the effectiveness of professional codes of ethics and corporate policies 
that attempt to deal with ethical dilemmas faced by business consultants. A mere codification of ethical values of the 
profession or organization may not counteract ethical ambivalence created and maintained through reward systems. 
The results suggest that unless ethical codes and policies are consistently reinforced with a significant reward and pun- 
ishment structure and truly integrated into the business culture, these mechanisms would be of limited value in actually 
regulating unethical conduct.'* 


Federal, state, and local governments also have laws, policies, and procedures in place to regulate 
research on human beings. The U.S. government began a process that covers all research having federal 
support. Initially implemented in 1966, the Institutional Review Boards (IRBs) engage in a risk assess- 
ment and benefit analysis review of proposed research. The Department of Health and Human Services 
(HHS) translated the federal regulations into policy. Most other federal and state agencies follow the 
HHS-developed guidelines. Exhibit 2-4 describes some characteristics of the IRB process. 

The current review requirement has been relaxed so that research that is routine no longer needs 
to go through the complete process.'* Each institution receiving funding from HHS or doing research 
for HHS is required to have its own IRB to review research proposals. Many institutions require all 
research, whether funded or unfunded by the government, to undergo review by the local IRB. The 
IRBs concentrate on two areas. First is the guarantee of obtaining complete, informed consent from 
participants. This can be traced to the first of 10 points in the Nuremberg Code.'° Complete informed 
consent has four characteristics: 


1. The participant must be competent to give consent. 
2. Consent must be voluntary. 
3. Participants must be adequately informed to make a decision. 


4. Participants should know the possible risks or outcomes associated with the research. 


The second item of interest to the IRB is the risk assessment and benefit analysis review. In the re- 
view, risks are considered when they add to the normal risk of daily life. Significantly, the only benefit 
considered is the immediate importance of the knowledge to be gained. Possible long-term benefits 
from applying the knowledge that may be gained in the research are not considered."” 

Other federal legislation that governs or influences the ways in which research is carried out are the 
Right to Privacy laws. Public Law 95-38 is the Privacy Act of 1974. This was the first law guarantee- 
ing Americans the right to privacy. Public Law 96-440, the Privacy Protection Act of 1980, carries the 
right to privacy further. These two laws are the basis for protecting the privacy and confidentiality of 
the participants and the data. 


> Resources for Ethical Awareness 


There is optimism for improving ethical awareness. According to the Center for Business Ethics at 
Bentley College, over a third of Fortune 500 companies have ethics officers, a substantial rise. Almost 
90 percent of business schools have ethics programs, up from a handful several years ago.'* Exhibit 2-5 
provides a list of recommended resources for business students, researchers, and managers. The Center 
for Ethics and Business at Loyola Marymount University provides an online environment for discuss- 
ing issues related to the necessity, difficulty, costs, and rewards of conducting business ethically. Its 
website offers a comprehensive list of business and research ethics links." 
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Exhibit 2-4 Characteristics of the Institutional Review Board (IRB) Process 


Process 


Examples 


Purpose 


Applications 


Initial Review 


Prepare IRB 
Materials 


Continuing 
Review 


Revision 


IRB Actions 


A committee is established to review and 
approve research involving human subjects. 


Determine the level of participant involvement in 
your study and select appropriate forms. 


Center level review (previously exempt research). 


Expedited review. 


Full review. 


An IRB submission form and research protocol. 


Research approved for limited period of time 
(e.g., one year). 


The principal investigator submits in writing any 
changes he or she intends to make to the 
study to the IRB. 


Approve 
Pending 


Disapprove 


Review minimizes potential research-related risks and requires 
full disclosure so that participants can make informed deci- 
sions about whether or not to participate. 


e Involves human subjects. 

¢ Does not involve human subjects. 
e Analyzes coded (secondary) data. 
« Analyzes biological specimens. 


e Research involving watching public behavior of children, in 
which the investigator does not take part in the activities, can 
be reviewed at the center level. 


* Research conducted in established or commonly accepted 
educational settings, involving normal educational practices, 
such as (1) research on regular and special education in- 
structional strategies, or (2) research on the effectiveness of 
or the comparison among instructional techniques, curricula, 
or classroom management methods. (Surveys or interviews 
cannot be center level reviewed.) 


Research on individual or group behavior characteristics (such 
as studies of perception, motivation, communication, cultural 
beliefs or practices, and social behavior) or research em- 
ploying survey, interview, oral history, focus group, program 
evaluation, human factors evaluation or quality assurance 
methodologies when the research does not qualify for center 
level review. 


Research involving physically intrusive procedures; where previ- 
ous experience has been shown to create a potential of risk 
to subjects; that may result in a significant level of psycho- 

logical or physical stress. 


Submissions are often accompanied by: 
¢ Informed-consent forms. 
* Completed informed consent form checklists. 


¢ Evidence of approval by cooperative IRBs at other sites. 


¢ Data collection instruments. 


* Certification of translation for consents or instruments to be 
used with non-English-speaking subjects. 


¢ Brochure/recruitment materials. 


Beyond time period, research cannot continue without IRB 
approval: A continuation request is submitted. 


Revision to the research protocol (e.g., changes to the in- 
formed consent form, survey instruments used, or number 
and nature of subjects. 


Approve as submitted. 

Pending: (1) Researcher clarifies an aspect of the study, provides 
additional information, or discusses the potential risks and ben- 
efits of the study, or (2) makes minor changes to the informed 
consent document(s) or the research protocol. 


Disapprove: Proposed research places the participants at risks 
that outweigh the benefit or value of the gained knowledge; 
study raises unacceptable ethical questions. 


Source: Based on Nova Southeastern University’s review information: http://www.nova.edu/irb/process.html#init_review. 
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Exhibit 2-5 Resources for Ethical Awareness 


Journals and Magazines 


Alert!; http://alert.marketingresearch.org/ 
Business Ethics; www.business-ethics.com/. 
Business Ethics Quarterly; https://secure.pdcnet.org/pdc/bvdb.nsf/journal?openform&journal=pdc_beq. 


Business and Professional Ethics Journal, DePaul University; http://commerce.depaul.edu/ethics/research-teaching/publications/ 
business-professional-ethics-j.asp. 


Business and Society; (IASB Journal); http://bas.sagepub.com/. 


Business and Society Review; http://www.wiley.com/bw/journal.asp?ref=0045-3609. 


Business and Society Review, Center for Business Ethics at Bentley College; http://www.bentley.edu/cbe/. 
Electronic Journal of Business Ethics and Organizational Studies (EJBO); http://ejbo.jyu.fi/. 

Ethics Newsline; www.globalethics.org/newsline/. 

Ethikos; www.ethikos.com/. 

Journal of Business Ethics; www.springerlink.com/content/100281/. 

Journal of Business Ethics; http://www.ingentaconnect.com/content/klu/busi. 


Marketing Research; http://www.marketingoower.com/AboutAMA/Pages/AMA Publications/AMA Magazines/Marketing Research/ 
MarketingResearch.aspx. 


Research, Training, and Conferences 


Advanced Research Techniques Forum, American Marketing Association Chicago, IL (800 AMA-1150; www.marketingpower.com). 


Applied Research Ethics National Association (ARENA) Boston, MA (617-423-4412; www.primr.org/). 

Business ethics conferences, The Conference Board, New York, NY (212-759-0900; www.conference-board.org). 

Center for Business Ethics, Bentley College, Waltham, MA (781-891-2981; www.bentley.edu/cbe/). 

Center for Ethical Business Culture, University of St. Thomas, Minneapolis, MN (800-328 6819 Ext. 2-4120; www.cebcglobal.org/). 
Center for Ethics and Business, Loyola Marymount University, Los Angeles, CA (810-338-2700; www.ethicsandbusiness.org). 


Center for Professional and Applied Ethics, University of North Carolina at Charlotte, Charlotte, NC (704-687-2850; 3542 
http://ethics.uncc.edu/). 


Center for the Study of Ethics in the Professions, Illinois Institute of Technology, Chicago, IL (812-567-3017; www.iit.edu/). 
Council of American Survey Research Organization (CASRO), Port Jefferson, NY (631-928-6954; www.casro.org). 

Dartmouth College Ethics Institute, Hanover, NH (603-646-1263; www.dartmouth.edu/~ethics/). 

Edmond J. Safra Foundation Center for Ethics, Harvard University, Cambridge, MA (617-495-1336; www.ethics.harvard.edu/). 


Electronic Privacy Information Center, Washington, DC (202-483-1140; www.epic.org). 


Ethics Corps Training for Business Leaders, Josephson Institute of Ethics, Marina del Rey, CA (310-306-1868; www.josephsoninstitute.org). 
Ethics Resource Center, Arlington, VA 703-647-2185 www.ethics.org). 
European Business Ethics Network Pamplona, Spain (84-948-425-600 ext. 2489; www.eben-net.org). 


Graduate Research Ethics Education Workshop, Association of Practical and Professional Ethics, Indiana University, Bloomington, IN 
(812-855-6450; http://www.indiana.edu/~appe/). 


Institute for Business and Professional Ethics, DePaul University, Chicago, IL (312-362-6624; http://commerce.depaul.edu/ethics/). 
International Association for Business and Society (www.iabs.net/). 
Marketing Research Association, Rocky Hill, CT (860-257-4008; www.marketingresearch.org). 


Markkula Center for Applied Ethics, Santa Clara University, Santa Clara, CA (408-554-5319; http://www.scu.edu/ethics/). 
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Research, Training, and Conferences (cont.) 


The Carol and Lawrence Ziklin Center for Business Ethics Research, The Wharton School, University of Pennsylvania, Philadelphia, A 


(215-898-1166; www.zicklincenter.org/). 


World Association of Public Opinion Research (WAPOR), Lincoln, NE (402-472-7720; http://wapor.unl.edu/). 
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Ethics are norms or standards of behavior that guide moral 
choices about our behavior and our relationships with oth- 
ers. Ethics differ from legal constraints, in which generally 
accepted standards have defined penalties that are univer- 
sally enforced. The goal of ethics in research is to ensure 
that no one is harmed or suffers adverse consequences 
from research activities. 

As research is designed, several ethical considerations 
must be balanced: 


e Protect the rights of the participant or subject. 


e Ensure the sponsor receives ethically conducted and 
reported research. 


e Follow ethical standards when designing research. 
e Protect the safety of the researcher and team. 


e Ensure the research team follows the design. 


De) 


In general, research must be designed so that a participant 
does not suffer physical harm, discomfort, pain, embarrass- 
ment, or loss of privacy. Begin data collection by explaining 
to participants the benefits expected from the research. 
Explain that their rights and well-being will be adequately 
protected and say how that will be done. Be certain that 
interviewers obtain the informed consent of the participant. 


The use of deception is questionable; when it is used, de- 
brief any participant who has been deceived. 


Many sponsors wish to undertake research without re- 
vealing themselves. Sponsors have the right to demand 
and receive confidentiality between themselves and the 
researchers. Ethical researchers provide sponsors with the 
research design needed to solve the managerial question. 
The ethical researcher shows the data objectively, despite 
the sponsor’s preferred outcomes. 

The research team’s safety is the responsibility of the 
researcher. Researchers should require ethical compliance 
from team members in following the research design, just as 
sponsors expect ethical behavior from the researcher. 


Many corporations and research firms have adopted a code 
of ethics. Several professional associations have detailed 
research provisions. Of interest are the American Associa- 
tion for Public Opinion Research, the American Marketing 
Association, the American Political Science Association, the 
American Psychological Association, and the American So- 
ciological Association. Federal, state, and local governments 
have laws, policies, and procedures in place to regulate 
research on human beings. 


ra 63 ACS 


code of ethics 40 
confidentiality 34 nondisclosure: 
debriefing 32 findings 37 
deception 30 purpose 37 


ethics 28 sponsor 36 


informed consent 31 


right to privacy 34 
right to quality 33 
right to safety 39 
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Making Research Decisions 

1 A Competitive Coup in the In-Flight Magazine. When 
the manager for market intelligence of AutoCorp, a major 
automotive manufacturer, boarded the plane in Chicago, 
her mind was on shrinking market share and late product 
announcements. As she settled back to enjoy the remains 
of a hectic day, she reached for the in-flight magazine. It 
was jammed into the seat pocket in front of her. 

Crammed into this already tiny space was a report with a 
competitor's logo, marked “Confidential—Restricted Circula- 
tion.” It contained a description of new product announce- 
ments for the next two years. Not only was it intended for a 
small circle of senior executives, but it also answered the ques- 
tions she had recently proposed to an external research firm. 

The proposal for the solicited research could be can- 
celed. Her research budget, already savaged, could be 
saved. She was home free, legally and career-wise. 

She foresaw only one problem. In the last few months, 
AutoCorp’s newly hired ethicist had revised the firm’s Busi- 
ness Conduct Guidelines. They now required company em- 
ployees in possession of a competitor’s information to return 
it or face dismissal. But it was still a draft and not formally 
approved. She had the rest of the flight to decide whether to 
return the document to the airline or slip it into her briefcase. 


a What are the most prudent decisions she can make 
about her responsibilities to herself and others? 


b What are the implications of those decisions even if there 
is no violation of law or regulation? 


2 Free Waters in Miro Beach: Boaters Inc. versus City 
Government.” The city commissioners of Miro Beach pro- 
posed limits on boaters who anchor offshore in waterfront 
areas of the St. Lucinda River adjoining the city. Residents 
had complained of pollution from the live-aboard boaters. 
The parking lot of boats created an unsightly view. 

The city based its proposed ordinance on research done 
by the staff. The staff did not hold graduate degrees in either 
public or business administration, and it was not known if staff 
members were competent to conduct research. The staff re- 
quested a proposal from a team of local university professors 
who had conducted similar work in the past. The research 
cost was $10,000. After receiving the proposal, the staff 
chose to do the work itself and not expend resources for the 
project. Through an unidentified source, the professors later 
learned their proposal contained enough information to guide 
the city’s staff and suggested data collection areas that might 
provide information that could justify the boaters’ claims. 

Based on the staff's one-time survey of waterfront litter, 
“oump-out” samples, and a weekly frequency count of boats, 
an ordinance was drafted and a public workshop was held. 
Shortly after, a group of concerned boat owners formed Boaters 
Inc., an association to promote boating, raise funds, and lobby 
the commission. The group’s claims were that the boaters (1) 


spent thousands of dollars on community goods and services, 
(2) did not create the litter, and (8) were being unjustly penalized 
because the commission’s fact finding was flawed. 

With the last claim in mind, the boaters flooded the city 
with public record requests. The clerks reported that some 
weeks the requests were one per day. Under continued 
pressure, the city attorney hired a private investigator (Pl) to 
infiltrate Boaters Inc. to collect information. He rationalized 
this on the grounds that the boaters had challenged the 
city’s grant applications in order to “blackmail the city into 
dropping plans to regulate the boaters.” 

The PI posed as a college student and worked for a time 
in the home of the boater organization’s sponsor while helping 
with mailings. Despite the Pl’s inability to corroborate the city 
attorney’s theory, he recommended conducting a background 
investigation on the organization’s principal, an employee of a 
tabloid newspaper. (The FBI, on request of city or county police 
organizations, generally performs background investigations.) 

The PI was not a boating enthusiast and soon drew sus- 
picion. Simultaneously, the organization turned up the heat 
on the city by requesting what amounted to 5,000 pages of 
information—“studies and all related documents containing 
the word ‘boat.’” Failing to get a response from Miro Beach, 
the boaters filed suit under the Florida Public Records Act. 
By this time, the city had spent $20,000. 

The case stalled, went to appeal, and was settled in favor 
of the boaters. A year later, the organization’s principal filed an 
invasion of privacy and slander suit against the city attorney, the 
PI, and the Pl’s firm. After six months, the suit was amended to 
include the city itself and sought $1 million in punitive damages. 


a What are the most prudent decisions the city can make 
about its responsibilities to itself and others? 


b What are the implications of those decisions even if there 
is no violation of law or regulation? 


The High Cost of Organizational Change. It was his first 
year of college teaching, and there were no summer teach- 
ing assignments for new hires. But the university was kind 
enough to steer him to an aviation firm, Avionics Inc., which 
needed help creating an organizational assessment survey. 
The assignment was to last five weeks, but it paid about the 
same as teaching all summer. The work was just about as 
perfect as it gets for an organizational behavior specialist. 
Avionics Inc.’s vice president, whom he met the first day, 
was cordial and smooth. The researcher would report to a 
senior manager who was coordinating the project with the 
human resources and legal departments. 

It was soon apparent that in the 25-year history of 
Avionics Inc., there had never been an employee survey. 
This was understandable given management’s lack of 
concern for employee complaints. Working conditions had 
deteriorated without management intervention, and govern- 
ment inspectors counted the number of heads down at 


desks as an index of performance. To make matters worse, 
the engineers were so disgruntled that word of unioniza- 
tion had spread like wildfire. A serious organizing effort was 
planned before the VP could approve the survey. 

Headquarters dispatched nervous staffers to monitor 
the situation and generally involve themselves with every as- 
pect of the questionnaire. Shadowed, the young researcher 
began to feel apprehension turn to paranoia. He consoled 
himself, however, with the goodwill of 500 enthusiastic, co- 
operative employees who had pinned their hopes for a better 
working environment to the results of this project. 

The data collection was textbook perfect. No one had 
asked to preview the findings or had shown any particular in- 
terest. In the fifth week, he boarded the corporate jet with the 
VP and senior manager to make a presentation at headquar- 
ters. Participants at the headquarters location were invited to 
attend. Management was intent on heading off unionization 
by showing its confidence in the isolated nature of “a few en- 
gineers’ complaints.” They had also promised to engage the 
participants in action planning over the next few days. 

An hour into the flight, the Avionics Inc. VP turned from 
his reading to the young researcher and said, “We have seen 
your results, you Know. And we would like you to change 
two key findings. They are not all that critical to this round of 
fixing the ‘bone orchard,’ and you'll have another crack at it 
as a real consultant in the fall.” 

“But that would mean breaking faith with your employees 
. .. people who trusted me to present the results objectively. 
It’s what | thought you wanted . . .” 

“Yes, well, look at it this way,” replied the VP. “All of your 
findings we can live with except these two. They’re an embar- 
rassment to senior management. Let me put it plainly. We have 
government contracts into the foreseeable future. You could 
retire early with consulting income from this place. Someone will 
meet us on the runway with new slides. What do you say?” 

a What are the most prudent decisions Avionics Inc. can 
make about its responsibilities to itself and others? 

b What are the implications of those decisions even if there 
is no violation of law or regulation? 

Data-Mining Ethics and Company Growth Square Off. 

SupplyCo. is a supplier to a number of firms in an indus- 

try. This industry has a structure that includes suppliers, 
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manufacturers, distributors, and consumers. Several com- 
panies are involved in the manufacturing process—from 
processed parts to creation of the final product—with each 
firm adding some value to the product. 

By carefully mining its customer data warehouse, 
SupplyCo. reveals a plausible new model for manufacturing 
and distributing industry products that would increase the 
overall efficiency of the industry 
system, reduce costs of produc- 
tion (leading to greater indus- 
try profits and more sales for 
SupplyCo.), and result in greater sales and profits for some 
of the industry’s manufacturers (SupplyCo.’s customers). 

On the other hand, implementing the model would hurt 
the sales and profits of other firms that are also SupplyCo.’s 
customers but which are not in a position (due to manpower, 
plant, or equipment) to benefit from the new manufacturing/ 
distribution model. These firms would lose sales, profits, and 
market share and potentially go out of business. 

Does SupplyCo. have an obligation to protect the inter- 
ests of all its customers and to take no action that would 
harm any of them, since SupplyCo. had the data within its 
warehouse only because of its relationship with its custom- 
ers? (It would betray some of its customers if it were to use 
the data in a manner that would cause these customers 
harm.) Or does it have a more powerful obligation to its 
stockholders and employees to aggressively pursue the new 
model that research reveals would substantially increase its 
sales, profits, and market share against competitors? 


The scenario in the Cummins 
Engines video case has 
some of the same properties 
as this ethical dilemma. 


a What are the most prudent decisions SupplyCo. Can 
make about its responsibilities to itself and others? 


b What are the implications of those decisions even if there 
is no violation of law or regulation? 


From the Headlines 
5 Hearsay Social is a company developed to monitor how 


workers at large companies interact with customers on 
Facebook, LinkedIn, and other social media sites. Compa- 
nies pay $100,000 or more for a digital dashboard that alerts 
supervisory managers if employees are violating privacy poli- 
cies, regulatory compliance rules, or other company policies. 
If you were a large client, like Farmers Insurance, what would 
you want built into the dashboard? 


_____ 


ee Akron Children’s Hospital 


ee Cummins Engines 


Proofpoint: Capitalizing on a Reporter’s Love of Statistics 


“You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 
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learningobjectives 
After reading this chapter, you should understand... 


The terminology used by professional researchers employing scientific thinking. 
What you need to formulate a solid research hypothesis. 


The need for sound reasoning to enhance business research results. 


6 6 Thought leaders, academics, business authors from the 
increasingly merging worlds of psychology, philosophy, 
economics, trends, semiotics are powerful forces, that 
have the ability to capture the imagination and attention 
at the highest level of business. Research needs to define 


itself more broadly, embrace the most exciting thinkers 
into our fold. We need to infuse our thinking with that of 
related fields. ® 9 


Edward Appleton, senior European consumer insights manager, 


Avery Dennison 
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“Myra, have you had any experience with research 
suppliers?” asks Jason. 

“Some. Actually, I worked for one of your 
competitors for a short time after college, on a project 
with the U.S. Army. That project helped me decide that 
research wasn’t my life’s work—not that it wasn’t and 
isn’t an important field and an important part of my 
new responsibilities.” 

“No need to apologize. Some of us have what it 
takes and others don’t.” 

“Actually, there wasn’t anything missing in my 
ability to observe data, or build rapport with study 
participants, or find insights,” shares Myra. “The project 
made all the papers; you probably read about it.” 

“Refresh my memory.” 

“The death rate near one Army munitions testing 
area was unexplainably high. Local activists were 
trying to shut it down, fearing it was an environmental 
hazard. The Army had a vested interest in keeping it 
open. Besides, it didn’t think the civilian deaths had 
anything to do with the firing range. U.S. Senator Sly 
forced the Army to investigate. Since the Army thought 
it had a public relations job on its hands, my firm was a 
logical choice; PR campaigns were a specialty. 

“The firing range was a played-out mine, strip- 
mined until it was worse than a moonscape. The area 
had once been a prosperous mining region, where the 
people were known for fearlessly and proudly going out 
to dig and produce. The nearest town was so severely 
economically depressed that, for the pitifully few jobs 
the Army provided, the folks welcomed the military in 
to bomb their backyard to cinders. 


“The cannon the Army was testing was impressive. 
Troops armed it with 3-inch shells, put on ear protectors 
and goggles, and lobbed shells into the range. There 
would be a tremendous flash and boom, and the shells 
would go roaring and soaring out of sight. We would 
soon hear a tremendous boom coming back to us and 
see dust and ash kicked up several hundred feet. We 
were all very happy not to be downrange. When we 
went downrange later, we found a huge crater and a 
fused puddle of iron, but nothing else but slag and 
molten rocks. 

“There was one problem. About every 20th shell 
would be a dud. It would fly off and land, and maybe 
kick up some dust, but explode it would not. 

“On paper, this was not supposed to be a problem. 
The Army sent an officious second lieutenant to brief us. 
He showed us reports that the Army had dropped such 
duds from hundred-foot platforms, from helicopters, 
had applied torches to them—everything—and had 
discovered them to be completely inert. The only thing 
he claimed would ignite one of these duds was to drop 
another live bomb on it. 

“Regrettably, this proved not to be the case. My 
team had barely finished its initial briefing when in 
the middle of the night we heard one of these so-called 
duds explode. We rushed out at dawn and, sure enough, 
found a new crater, molten slag, molten rock, and so 
forth. It was quite a mystery. 

“Our team took shifts doing an all-night observation 
study. During my two-hour stint, my partner and I saw 
people with flashlights moving around in there. 


“We didn’t know if the people were military or 
civilian. We learned later that locals were coming 
in at night, intending to crack open the bombs and 
scavenge for copper wire or anything they thought 
was salvageable. Except, of course, their actions 
occasionally ignited one of the beauties and erased any 
evidence of a crime being committed by vaporizing the 
perpetrators on the spot. 

“Part of our research was to measure public 
sentiment about the firing range among the locals. 
During our stay in the area, we discovered the locals 
were involved in every kind of thrill sport. It was 
not unusual to see a 50-mile auto race with four 
ambulances on hand on the edge of the oval, to cart off 
the carnage to the surgical hospital in the next county. 
I saw men leap into cars with threadbare tires, loose 
wheels, malfunctioning brakes, with brake fluid and 
transmission fluid drooling all over the track. They 
could wheel their cars out onto the track on a tire they 
knew was thin as tissue, and if it blew out and put them 
in the hospital, their reaction was ‘Some days you can’t 
win for losin’.” Nobody thought anything of this. If we 
asked, their answer was, ‘I'll go when my number is 
up,’ or ‘It’s not in my hands.’ 

“Their attitude made sense, from a cultural- 
economic view. That attitude had permitted the men 
to go down in the mines year after year. Even the local 
sheriff wouldn’t stop their daredevil behavior. “They 
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are going to die anyway,’ he was overheard remarking. 
‘We all are going to die. People die every month that 
never go out on that dirt track.’ Of course, unlike 
driving a car, messing with a potentially live bomb 
didn’t leave much to skill but left everything to chance. 

alice 
campaign to keep the scavengers out but, given our 


Army had considered an _ educational 


findings, decided it couldn’t deal with such thinking 
by applying logic. Instead, it changed its procedure. 
The troops would now fire the shells in the morning 
and spend the afternoon finding the duds, to which 
they attached kerosene lanterns. At dusk, a fighter- 
bomber would fly over the area and bomb the 
lanterns—and the duds—to a molecular state. It was 
neat and it worked. And the death rate of the locals 
dropped dramatically.” 

As Myra finished her story, Jason asks, “It sounds like 
a successful project. By studying the locals’ attitudes 
and behavior, you could discard the alternative of the 
education campaign. Why did you decide research 
wasn’t for you?” 

“My boss didn’t like the idea that I broke 
confidentiality and told a local reporter what the locals 
were doing. I’d seen someone’s dad or brother blown 
to pieces and felt I had to act. My dismissal taught me 
one of the rules of good research—the client always 
gets to choose whether to use, or release, the findings 
of any study.” 


When we do research, we seek to know what is in order to understand, explain, and predict phenomena. 
We might want to answer the question “What will be the department’s reaction to the new flexible work 
schedule?” or “Why did the stock market price surge higher when all normal indicators suggested it 
would go down?” When dealing with such questions, we must agree on definitions. Which members 
of the department: clerical or professional? What kind of reaction? What are normal indicators? These 
questions require the use of concepts, constructs, and definitions. 


To understand and communicate information about objects and events, there must be a common ground 
on which to do it. Concepts serve this purpose. A concept is a generally accepted collection of mean- 
ings or characteristics associated with certain events, objects, conditions, situations, and behaviors. 
Classifying and categorizing objects or events that have common characteristics beyond any single 
observation creates concepts. When you think of a spreadsheet or a warranty card, what comes to mind 
is not a single example but your collected memories of all spreadsheets and warranty cards, from which 
you abstract a set of specific and definable characteristics. 
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The smart phone has dramatically affected the way we make purchases in stores, as noted by results from a study by Interac- 
tive Advertising Bureau. What are the concepts and constructs involved in this study? What might be the operational defini- 
tions? What are the variables? What hypotheses would you propose about smart phone use and shopping behavior? 
www.iab.net 


We abstract such meanings from our experiences and use words as labels to designate them. For 
example, we see a man passing and identify that he is running, walking, skipping, crawling, or hopping. 
These movements all represent concepts. We also have abstracted certain visual elements by which we 
identify that the moving object is an adult male, rather than an adult female or a truck or a horse. We 
use numerous concepts daily in our thinking, conversing, and other activities. 


Sources of Concepts 


Concepts that are in frequent and general use have been developed over time through shared language 
usage. We acquire them through personal experience. Ordinary concepts make up the bulk of commu- 
nication even in research, but we often run into difficulty trying to deal with an uncommon concept or a 
newly advanced idea. One way to handle this problem is to borrow from other languages (e.g., gestalt) or 
to borrow from other fields (e.g., from art, impressionism). The concept of gravitation is borrowed from 
physics and used in marketing in an attempt to explain why people shop where they do. The concept of 
distance is used in attitude measurement to describe degree of variability between the attitudes of two or 
more persons. Threshold is used effectively to describe a concept about the way we perceive. 

Sometimes we need to adopt new meanings for words (make a word cover a different concept) or de- 
velop new labels for concepts. The recent broadening of the meaning of model is an example of the first in- 
stance; the development of concepts such as sibling and status-stress is an example of the second. When we 
adopt new meanings or develop new labels, we begin to develop a specialized jargon or terminology. Jargon 
no doubt contributes to efficiency of communication among specialists, but it excludes everyone else. 


Importance to Research 


In research, special problems grow out of the need for concept precision and inventiveness. We design hy- 
potheses using concepts. We devise measurement concepts by which to test these hypothetical statements. 
We gather data using these measurement concepts. The success of research hinges on (1) how clearly we 
conceptualize and (2) how well others understand the concepts we use. For example, when we survey people 
on the question of customer loyalty, the questions we use need to tap faithfully the attitudes of the partici- 
pants. Attitudes are abstract, yet we must attempt to measure them using carefully selected concepts. 
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The challenge is to develop concepts that others will clearly understand. We might, for example, ask 
participants for an estimate of their family’s total income. This may seem to be a simple, unambiguous 
concept, but we will receive varying and confusing answers unless we restrict or narrow the concept 
by specifying: 


e Time period, such as weekly, monthly, or annually. 

° Before or after income taxes. 

¢ For head of family only or for all family members. 

° For salary and wages only or also for dividends, interest, and capital gains. 


e Income in kind, such as free rent, employee discounts, or food stamps. 


Constructs 


Concepts have progressive levels of abstraction—that is, the degree to which the concept does or does 
not have something objective to refer to. Table is an objective concept. We can point to a table, and 
we have images of the characteristics of all tables in our mind. An abstraction like personality is much 
more difficult to visualize. Such abstract concepts are often called constructs. A construct is an image 
or abstract idea specifically invented for a given research and/or theory-building purpose. We build 
constructs by combining the simpler, more concrete concepts, especially when the idea or image we in- 
tend to convey is not subject to direct observation. When Jason and Myra tackle MindWriter’s research 
study, they will struggle with the construct of satisfied service customer. 

Concepts and constructs are easily confused. Consider this example: Heather is a human resource 
analyst at CadSoft, an architectural software company that employs technical writers to write product 
manuals, and she is analyzing task attributes of a job in need of redesign. She knows the job description 
for technical writer consists of three components: presentation quality, language skill, and job interest. 
Her job analysis reveals even more characteristics. 

Exhibit 3-1 illustrates some of the concepts and constructs Heather is dealing with. The concepts 
at the bottom of the exhibit (format accuracy, manuscript errors, and keyboarding speed) are the most 
concrete and easily measured. We are able to observe keyboarding speed, for example, and even with 
crude measures agree on what constitutes slow and fast keyboarders. Keyboarding speed is one concept 
in the group that defines a construct that the human resource analyst calls “presentation quality.” Pre- 
sentation quality is really not directly observable. It is a nonexistent entity, a “constructed type,” used 
to communicate the combination of meanings presented by the three concepts. Heather uses it only as 
a label for the concepts she has discovered are related empirically. 

Concepts at the next level in Exhibit 3-1 are vocabulary, syntax, and spelling. Heather also finds 
them to be related. They form a construct that she calls “language skill.” She has chosen this term 
because the three concepts together define the language requirement in the job description. Language 
skill is placed at a higher level of abstraction in the exhibit because two of the concepts it comprises, 
vocabulary and syntax, are more difficult to observe and their measures are more complex. 

Heather has not yet measured the last construct, “job interest.” It is the least observable and the most 
difficult to measure. It will likely be composed of numerous concepts—many of which will be quite 
abstract. Researchers sometimes refer to such entities as hypothetical constructs because they can 
be inferred only from the data; thus, they are presumed to exist but must await further testing to see 
what they actually consist of. If research shows the concepts and constructs in this example to be inter- 
related, and if their connections can be supported, then Heather will have the beginning of a conceptual 
scheme. In graphic form, it would depict the relationships among the knowledge and skill requirements 
necessary to clarify the job redesign effort. 


Definitions 


Confusion about the meaning of concepts can destroy a research study’s value without the researcher 
or client even knowing it. If words have different meanings to the parties involved, then the parties are 
not communicating well. Definitions are one way to reduce this danger. 
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>Exhibit 3-1 Constructs Composed of Concepts in a Job Redesign 
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Researchers struggle with two types of definitions: dictionary definitions and operational defini- 
tions. In the more familiar dictionary definition, a concept is defined with a synonym. For example, 
a customer is defined as a patron; a patron, in turn, is defined as a customer or client of an estab- 
lishment; a client is defined as one who employs the services of any professional and, loosely, as 
a patron of any shop.' Circular definitions may be adequate for general communication but not 
for research. In research, we measure concepts and constructs, and this requires more rigorous 
definitions. 


Operational Definitions 


An operational definition is a definition stated in terms of specific criteria for testing or measurement. 
These terms must refer to empirical standards (i.e., we must be able to count, measure, or in some other 
way gather the information through our senses). Whether the object to be defined is physical (e.g., a 
can of soup) or highly abstract (e.g., achievement motivation), the definition must specify the charac- 
teristics and how they are to be observed. The specifications and procedures must be so clear that any 
competent person using them would classify the object in the same way. 

During her research project with the military, Myra observed numerous shells that, when fired, did 
not explode on impact. She knew the Army attached the operational definition “a shell that does not ex- 
plode on impact” to the construct dud shell. But if asked, Myra would have applied the operational term 
dud shell only to “a shell that, once fired from a cannon, could not be made to explode by any amount of 
manipulation, human or mechanical.” Based on her operational definition, the town’s residents rarely 
encountered “duds” during their excursions onto the firing range. 

Suppose college undergraduates are classified by class. No one has much trouble understanding 
such terms as freshman, sophomore, and so forth. But the task may not be that simple if you must de- 
termine which students fall in each class. To do this, you need operational definitions. 
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Using Scientific Definitions to Shape Political Debate over BioMed 


When politics trumps science in defining critical research termi- Another example relates to the Kansas House of Represen- 
nology, legislators intentionally or unwittingly fail to communicate tatives, which had two bills in process. One claimed “to define 
information accurately. This could be critical in discovering prod- terms related to human cloning.” The companion bill sought to 
ucts for breakthrough cures, tracking the progression of diseases ban public funding for somatic cell nuclear transfer (SCNT), the 
(thus affecting decisions about hospital staffing and insurance), bill’s term for creating cloned embryonic stem cells. Opponents 
and finding better ways to test new drugs to discover their various of embryonic stem-cell research (who contend that embryos are 
applications. human beings and wish to ban such research) found the defini- 

One example is The National Academies, which advises the tion credible. Citing “68 percent of Kansans support somatic cell 
federal government and public on scientific issues. It has “cre- nuclear transfer, there is also strong opposition to reproductive 
ated voluntary guidelines for embryonic stem cell research.” cloning.” Thus, by combining both techniques in the public’s 


These guidelines also “provide a comprehensive definition mind in a single operational definition, opponents aim to ban 
of terms that are accepted by every major research body in SCNT. 

the U.S.” Because stem-cell research and human cloning are Paul Terranova, vice chancellor for research at Kansas Uni- 
such volatile political issues, the federal government hasn’t versity Medical Center, is critical of the many scientific inaccura- 
proposed countrywide guidelines. As a result, individual states cies in the definitions used in both bills. When politics collide with 
opportunistically exploit scientific terminology and fill the void science, should politics triumph? 

with altered definitions that operationally join humans with em- 
bryos and add overarching definitions of human cloning. 


www.kumc.edu; www.kslegislature.org 


Operational definitions may vary, depending on your purpose and the way you choose to measure 
them. Here are two different situations requiring different definitions of the same concepts: 


1. You conduct a survey among students and wish to classify their answers by their class levels. 
You merely ask them to report their class status and you record it. In this case, class is 
freshman, sophomore, junior, or senior; and you accept the answer each respondent gives as 
correct. This is a rather casual definition process but nonetheless an operational definition. It is 
probably adequate even though some of the respondents report inaccurately. 


2. You make a tabulation of the class level of students from the university registrar’s annual 
report. The measurement task here is more critical, so your operational definition needs to be 
more precise. You decide to define class levels in terms of semester hours of credit completed 
by the end of the spring semester and recorded in each student’s record in the registrar’s office: 


Freshman Fewer than 30 hours’ credit 
Sophomore 30 to 59 hours’ credit 
Junior 60 to 89 hours’ credit 
Senior 90 or more hours’ credit 


Those examples deal with relatively concrete concepts, but operational definitions are even more 
critical for treating abstract ideas. Suppose one tries to measure a construct called “consumer socializa- 
tion.” We may intuitively understand what this means, but to attempt to measure it among consumers 
is difficult. We would probably develop questions on skills, knowledge, and attitudes; or we may use 
a scale that has already been developed and validated by someone else. This scale then operationally 
defines the construct. 

Whether you use a definitional or operational definition, its purpose in research is basically the 
same—to provide an understanding and measurement of concepts. We may need to provide operational 
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definitions for only a few critical concepts, but these will almost always be the definitions used to 
develop the relationships found in hypotheses and theories. 


Variables 


In practice, the term variable is used as a synonym for construct, or the property being studied. In this 
context, a variable is a symbol of an event, act, characteristic, trait, or attribute that can be measured 
and to which we assign values.’ 

For purposes of data entry and analysis, we assign numerical value to a variable based on the vari- 
able’s properties. For example, some variables, said to be dichotomous, have only two values, reflect- 
ing the presence or absence of a property: employed—unemployed or male-female have two values, 
generally 0 and 1. When Myra Wines observed the cannon shells, they were exploded or unexploded. 
Variables also take on values representing added categories, such as the demographic variables of race 
or religion. All such variables that produce data that fit into categories are said to be discrete, because 
only certain values are possible. An automotive variable, for example, where “Chevrolet” is assigned 
a5 and “Honda” is assigned a 6, provides no option for a 5.5. 

Income, temperature, age, and a test score are examples of continuous variables. These variables 
may take on values within a given range or, in some cases, an infinite set. Your test score may range 
from 0 to 100, your age may be 23.5, and your present income could be $35,000. The procedure for 
assigning values to variables is described in detail in Chapter 11. 


Independent and Dependent Variables 


Researchers are most interested in relationships among variables. For example, does a newspaper cou- 
pon (independent variable) influence product purchase (dependent variable), or can a salesperson’s 
ethical standards influence her ability to maintain customer relationships? As one writer notes: 


There’s nothing very tricky about the notion of independence and dependence. But there is something tricky about the 
fact that the relationship of independence and dependence is a figment of the researcher’s imagination until demon- 
strated convincingly. Researchers hypothesize relationships of independence and dependence: They invent them, and 
then they try by reality testing to see if the relationships actually work out that way.° 


Many textbooks use the term predictor variable as a synonym for independent variable (IV). This 
variable is manipulated by the researcher, and the manipulation causes an effect on the dependent vari- 
able. We recognize that there are often several independent variables and that they are probably at least 
somewhat “correlated” and therefore not independent among themselves. Similarly, the term criterion 
variable is used synonymously with dependent variable (DV). This variable is measured, predicted, 
or otherwise monitored and is expected to be affected by manipulation of an independent variable. 
Exhibit 3-2 lists some terms that have become synonyms for independent variable and dependent 
variable. 


>Exhibit 3-2 Independent and Dependent Variables: Synonyms 


Independent Variable Dependent Variable 


Predictor Criterion 


Stimulus Response 


Antecedent Consequence 
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In each relationship, there is at least one independent variable (IV) and one dependent variable 
(DV). It is normally hypothesized that, in some way, the IV “causes” the DV to occur. It should be 
noted, however, that although it is easy to establish whether an IV influences a DV, it is much harder 
to show that the relationship between an IV and DV is a causal relationship (see also Chapter 6). In 
Exhibit 3-3a, this relationship is illustrated by an arrow pointing from the independent variable to 
the dependent variable. For simple relationships, all other variables are considered extraneous and 
are ignored. 


Moderating or Interaction Variables 


In actual study situations, however, such a simple one-to-one relationship needs to be conditioned or 
revised to take other variables into account. Often, we can use another type of explanatory variable 
that is of value here: the moderating variable (MV). A moderating or interaction variable is a second 
independent variable that is included because it is believed to have a significant contributory or contin- 
gent effect on the original [V—DV relationship. The arrow pointing from the moderating variable to the 
arrow between the IV and DV in Exhibit 3-3a shows the difference between an IV directly impacting 


>Exhibit 3-3 Relationships among Types of Variables 


(a) 
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the DV and an MV affecting the relationship between an IV and the DV. For example, one might hy- 
pothesize that in an office situation: 


The introduction of a four-day working week (IV) will lead to higher productivity (DV), especially among younger workers (MV). 


In this case, there is a differential pattern of relationship between the four-day week and productivity 
that results from age differences among the workers. Hence, after introduction of a four-day working 
week, the productivity gain for younger workers is higher than that for older workers. It should be 
noted that the effect of the moderating or interaction variable is the “surplus” of the combined occur- 
rence of introducing a four-day working week and being a younger worker. For example, let’s assume 
that the productivity of younger workers is 12 percentage points higher than that for older workers, 
and that the productivity of workers having a four-day working week is 6 percentage points higher 
than those of workers having a five-day working week. If the productivity of a younger worker having 
a four-day working week is only 18 percentage points higher than the productivity of a older worker 
with a five-day working week, there is no interaction effect, because the 18 percentage points are the 
sum of the main effects. There would be an interaction effect if the productivity of the younger worker 
on a four-day week was, say, 25 percentage points higher than the productivity of the older worker on 
a five-day week. 

Whether a given variable is treated as an independent or moderating variable depends on the hy- 
pothesis under investigation. If you were interested in studying the impact of the length of the working 
week, you would make the length of week the IV. If you were focusing on the relationship between age 
of worker and productivity, you might use working week length as an MV. 


Extraneous Variables 


An almost infinite number of extraneous variables (EVs) exists that might conceivably affect a 
given relationship. Some can be treated as IVs or MVs, but most must either be assumed or excluded 
from the study. Fortunately, an infinite number of variables has little or no effect on a given situa- 
tion. Most can safely be ignored because their impact occurs in such a random fashion as to have 
little effect. Others might influence the DV, but their effect is not at the core of the problem we in- 
vestigate. Still, we want to check whether our results are influenced by them. Therefore, we include 
them as control variables (CVs) in our investigation to ensure that our results are not biased by not 
including them. Taking the example of the effect of the four-day working week again, one would 
normally think that weather conditions, the imposition of a local sales tax, the election of a new 
mayor, and thousands of similar events and conditions would have little effect on working week and 
office productivity. 

Extraneous variables can also be confounding variables (CFVs) to our hypothesized [V—DV re- 
lationship, similar to moderating variables. You may consider that the kind of work being done might 
have an effect on the impact of working week length on office productivity. This might lead you to in- 
troducing time spent in a meeting to coordinate the work as a confounding variable (CFV). In our office 
example, we would attempt to control for type of work by studying the effect of the four-day working 
week within groups attending meetings with different intensity. In Exhibit 3-3b, weather is shown as 
an extraneous variable; the broken line indicates that we included it in our research because it might 
influence the DV, but we consider the CV as irrelevant for the investigation of our research problem. 
Similarly we included the type of work as a CFV. 


Intervening Variables 


The variables mentioned with regard to causal relationships are concrete and clearly measurable—that 
is, they can be seen, counted, or observed in some way. Sometimes, however, one may not be com- 
pletely satisfied by the explanations they give. Thus, while we may recognize that a four-day working 
week results in higher productivity, we might think that this is not the whole story—that working 
week length affects some intervening variable (IVV) that, in turn, results in higher productivity. An 
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IVV is a conceptual mechanism through which the IV and MV might affect the DV. The IVV can be 
defined as a factor that theoretically affects the DV but cannot be observed or has not been measured; 
its effect must be inferred from the effects of the independent and moderator variables on the observed 
phenomenon.* 

In the case of the working week hypothesis, one might view the intervening variable (IVV) to be job 
satisfaction, giving a hypothesis such as: 


The introduction of a four-day working week (IV) will lead to higher productivity (DV) by increasing job satisfaction ( 


WV). 


Here we assume that a four-day working week increases job satisfaction; similarly, we can 
assume that attending internal meetings is an indicator negatively related to the routine character 
of work. Exhibit 3-3c illustrates how theoretical constructs, which are not directly observed, fit into 
our model. 


Propositions and Hypotheses 


We define a proposition as a statement about observable phenomena (concepts) that may be judged 
as true or false. When a proposition is formulated for empirical testing, we call it a hypothesis. As a 
declarative statement about the relationship between two or more variables, a hypothesis is of a tenta- 
tive and conjectural nature. 

Hypotheses have also been described as statements in which we assign variables to cases. A case is 
defined in this sense as the entity or thing the hypothesis talks about. The variable is the characteristic, 
trait, or attribute that, in the hypothesis, is imputed to the case.> For example, we might create the fol- 
lowing hypothesis: 


Brand Manager Jones (case) has a higher-than-average achievement motivation (variable). 
If our hypothesis was based on more than one case, it would be a generalization. For example: 


Brand managers in Company Z (cases) have a higher-than-average achievement motivation 
(variable). 


Descriptive Hypotheses 


Both of the preceding hypotheses are examples of descriptive hypotheses. They state the existence, 
size, form, or distribution of some variable. Researchers often use a research question rather than a 
descriptive hypothesis. For example: 


In Detroit (case), our potato chip market share What is the market share for our potato 
(variable) stands at 13.7 percent. chips in Detroit? 


Eighty percent of Company Z stockholders Do stockholders of Company Z favor an 
(cases) favor increasing the company’s cash increased cash dividend? 
dividend (variable). 
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Either format is acceptable, but the descriptive hypothesis format has several advantages: 


e It encourages researchers to crystallize their thinking about the likely relationships to be found. 
e It encourages them to think about the implications of a supported or rejected finding. 


¢ It is useful for testing statistical significance. 


Relational Hypotheses 


The research question format is less frequently used with a situation calling for relational hypotheses. 
These are statements that describe a relationship between two variables with respect to some case. For 
example, “Foreign (variable) cars are perceived by American consumers (case) to be of better quality 
(variable) than domestic cars.” In this instance, the nature of the relationship between the two variables 
(“country of origin” and “perceived quality’) is not specified. Is there only an implication that the vari- 
ables occur in some predictable relationship, or is one variable somehow responsible for the other? The 
first interpretation (unspecified relationship) indicates a correlational relationship; the second (predict- 
able relationship) indicates an explanatory, or causal, relationship. 

Correlational hypotheses state that the variables occur together in some specified manner without 
implying that one causes the other. Such weak claims are often made when we believe there are more 
basic causal forces that affect both variables or when we have not developed enough evidence to claim 
a stronger linkage. Here are three sample correlational hypotheses: 


Young women (under 35 years of age) purchase fewer units of our product than women who 
are 35 years of age or older. 


The number of suits sold varies directly with the level of the business cycle. 


People in Atlanta give the president a more favorable rating than do people in St. Louis. 


By labeling these as correlational hypotheses, we make no claim that one variable causes the other to 
change or take on different values. 

With explanatory (causal) hypotheses, there is an implication that the existence of or a change in 
one variable causes or leads to a change in the other variable. As we noted previously, the causal vari- 
able is typically called the independent variable (IV) and the other the dependent variable (DV). Cause 
means roughly to “help make happen.” So the IV need not be the sole reason for the existence of or 
change in the DV. Here are four examples of explanatory hypotheses: 


An increase in family income (IV) leads to an increase in the percentage of income saved (DV). 


Exposure to the company’s messages concerning industry problems (IV) leads to more 
favorable attitudes (DV) by employees toward the company. 


Loyalty to a particular grocery store (IV) increases the probability of purchasing the private 
brands (DV) sponsored by that store. 


An increase in the price of salvaged copper wire (IV) leads to an increase in scavenging (DV) 
on the Army firing range. 


In proposing or interpreting causal hypotheses, the researcher must consider the direction of in- 
fluence. In many cases, the direction is obvious from the nature of the variables. Thus, one would 
assume that family income influences savings rate rather than the reverse. This also holds true for the 
Army example. Sometimes our ability to identify the direction of influence depends on the research 
design. In the worker attitude hypothesis, if the exposure to the message clearly precedes the atti- 
tude measurement, then the direction of exposure to attitude seems clear. If information about both 
exposure and attitude was collected at the same time, the researcher might be justified in saying that 
different attitudes led to selective message perception or nonperception. Store loyalty and purchas- 
ing of store brands appear to be interdependent. Loyalty to a store may increase the probability of 
one’s buying the store’s private brands, but satisfaction with the store’s private brand may also lead 
to greater store loyalty. 
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Radio Chips versus Retinal Scans: Which Theory Offers the Best Protection? 


When the first confirmed case of bovine spongiform encepha- 
lopathy (BSE—known as “mad cow” disease) was discovered 
in a Washington state dairy cow in December 2003, numerous 
countries banned U.S. beef imports, bringing the $3.2 billion 
export industry to a standstill. That year, the United States De- 
partment of Agriculture (USDA) performed random tests on ap- 
proximately 0.03 percent of all slaughtered cattle, about 20,000 
cows of the nearly 40 million head of cattle slaughtered annually. 
In comparison, western European countries tested 10 million 
cows and Japan tested each of its 1.2 million slaughtered cows. 
Theories are essential to a researcher’s quest to explain and 
predict phenomena while creating business opportunities and 
informing public policy. One USDA theory is that the best way to 
identify sources of cattle-born disease is to monitor a cow from 
birth to slaughter. Thus, the USDA wanted a national livestock 
database. After evaluating the options, the USDA proposed an- 
other theory: Cows tagged with radio frequency identification 
devices (RFID) would create the most accurate database. 
About the size of a quarter, the RFID tag is stapled to the 
base of the animal’s ear. It is programmed with a numeric code 
that is scanned by a stationary or handheld device when a cow 
reaches a new location in the production process. As cows 
move from farm to feeding lot to slaughterhouse, each animal’s 
origin and location can be updated in the national database. 
But RFID tags can be damaged, dislodged, or tampered 
with. Slaughterhouses need additional safeguards to be certain 


The Role of the Hypothesis 


these devices don’t end up in the meat. “All you need is one 
chip in someone’s burger and you’ve got a problem,” says Brian 
Bolton, vice president of marketing for Optibrand. This Colorado 
company offers a different theory for the best identification and 
racking: A camera that records the unique vascular patterns in 
a cow’s retina at each stage of the beef production chain is the 
most reliable. With retinal scanning, Bolton says, “the tracking 
echnology is contained in the handheld reader. It takes a tiny 
picture of a cow’s retina and then links it to that animal’s com- 
puterized record.” Meatpacker Swift & Co., the nation’s third- 
argest beef processor, has been using Optibrand’s devices for 
several years. Retinal scan wands also read RFID tags, access 
global positioning receivers, and stamp each scan with a loca- 
ion record. However, retinal scanning is not always practical be- 
cause scans must be taken about an inch from an animal's eye. 

In addition to RFID and retinal scanning, beef producers and 
processors implement other tracking systems, thus implement- 
ing their own theories. Some use implantable computer chips 
and others use DNA matching systems. While still preferring 
RFID technology, the USDA’s director of national animal identifi- 
cation, John F. Wiemers, concedes, “We think there’s room for 
all these technologies.” 

Which tracking theory do you favor? What are the most im- 
portant variables you would consider in justifying your decision? 


www.usda.gov; www.optibrand.com; 
http://www.jbsswift.com 


In research, a hypothesis serves several important functions: 


e It guides the direction of the study. 


¢ It identifies facts that are relevant and those that are not. 


e It suggests which form of research design is likely to be most appropriate. 


It provides a framework for organizing the conclusions that result. 


Unless the researcher curbs the urge to include additional elements, a study can be diluted by trivial 
concerns that do not answer the basic questions posed by the management dilemma. The virtue of the 
hypothesis is that, if taken seriously, it limits what shall be studied and what shall not. To consider 
specifically the role of the hypothesis in determining the direction of the research, suppose we use this: 


Husbands and wives agree in their perceptions of their respective roles in purchase decisions. 


The hypothesis specifies who shall be studied (married couples), in what context they shall be studied 
(their consumer decision making), and what shall be studied (their individual perceptions of their roles). 

The nature of this hypothesis and the implications of the statement suggest that the best research 
design is a communication-based study, probably a survey or interview. We have at this time no other 
practical means to ascertain perceptions of people except to ask about them in one way or another. In 
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>Exhibit 3-4 Checklist for Developing a Strong Hypothesis 


Criteria Interpretation 


Adequate for lts Purpose (1 Does the hypothesis reveal the original problem condition? 
LI Does the hypothesis clearly identify facts that are relevant and those that are not? 


11 Does the hypothesis clearly state the condition, size, or distribution of some variable in terms of 
values meaningful to the research problem (descriptive)? 


11 Does the hypothesis explain facts that gave rise to the need for explanation (explanatory)? 


LI Does the hypothesis suggest which form of research design is likely to be most appropriate? 


LI Does the hypothesis provide a framework for organizing the conclusions that result? 


Better Than Its Rivals (1 Does the hypothesis explain more facts than its rivals? 


11 Does the hypothesis explain a greater variety or scope of facts than its rivals? 


LJ Is the hypothesis one that informed judges would accept as being the most likely? 


addition, we are interested only in the roles that are assumed in the purchase or consumer decision- 
making situation. The study should not, therefore, involve itself in seeking information about other 
types of roles husbands and wives might play. Reflection upon this hypothesis might also reveal that 
husbands and wives disagree on their perceptions of roles, but the differences may be explained in 
terms of additional variables, such as age, social class, background, personality, and other factors not 
associated with their difference in gender. 


What Is a Strong Hypothesis? A strong hypothesis should fulfill three conditions: 


e Adequate for its purpose. 
° Testable. 


¢ Better than its rivals. 


The conditions for developing a strong hypothesis are developed more fully in Exhibit 3-4. 


Theory 


Hypotheses play an important role in the development of theory. How theory differs from hypothesis 
may cause confusion. We make the general distinction that the difference between theory and hypoth- 
esis is one of degree of complexity and abstraction. In general, theories tend to be complex and abstract 
and to involve multiple variables. Hypotheses, on the other hand, tend to be more simple, limited- 
variable statements involving concrete instances. 

A person not familiar with research uses the term theory to express the opposite of fact. In this sense, 
theory is viewed as being speculative or “ivory tower.” One hears that managers need to be less theo- 
retical or that some idea will not work because it is too theoretical. This is an incorrect picture of the re- 
lationship between fact and theory to the researcher. In truth, fact and theory are each necessary for the 
other to be of value. Our ability to make rational decisions, as well as to develop scientific knowledge, 
is measured by the degree to which we combine fact and theory. We all operate on the basis of theories 
we hold. In one sense, theories are the generalizations we make about variables and the relationships 
among them. We use these generalizations to make decisions and predict outcomes. For example, it is 
midday and you note that the outside natural light is dimming, dark clouds are moving rapidly in from 
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>Exhibit 3-5 Traditional Product Life Cycle 


Stage of the product life cycle TT 


Introduction Growth Maturity Decline 


Total industry 
sales revenue 


Total industry 
profit 


Sales revenue or profit 


Source: Adapted from Roger Kerin, Eric Berkowitz, Steven Hartley, and William Rudelius, Marketing, 7th ed. (Burr Ridge, IL: McGraw-Hill, 
2003), p. 295. 


the west, the breeze is freshening, and the air temperature is cooling. Would your understanding of the 
relationship among these variables (your weather theory) lead you to predict that something decidedly 
wet will probably occur in a short time? 

A theory is a set of systematically interrelated concepts, definitions, and propositions that are ad- 
vanced to explain and predict phenomena (facts). In this sense, we have many theories and use them 
continually to explain or predict what goes on around us. To the degree that our theories are sound and 
fit the situation, we are successful in our explanations and predictions. 

In marketing, the product life cycle describes the stages that a product category goes through in 
the marketplace.° The generalized product life cycle has four stages (although the length and shape 
of product life cycles differ): introduction, growth, maturity, and decline (Exhibit 3-5). In each stage, 
many concepts, constructs, and hypotheses describe the influences that change revenue and profit. 
Definitions are also needed for communicating about the claims of the theory and its consistency in 
testing to reality. 

For example, in the growth stage, companies spend heavily on advertising and promotion to 
create product awareness. In the early period of this stage these expenditures may be made to fuel 
primary demand (construct), improving product class awareness rather than brand awareness. Also, 
high pricing may reflect skimming (concept) to help the company recover developmental costs. The 
product manager may alternatively use low pricing, or penetration pricing (concept), to build unit 
volume. In the growth stage, sales increase rapidly because many consumers are trying or actu- 
ally using the product; and those who tried (were satisfied) and bought again—repeat purchasers 
(concept)—are swelling the ranks. If the company is unable to attract repeat purchasers, this usu- 
ally means death for the product (proposition). The maturity stage is a good time for the company 
in terms of generating cash (proposition). The costs of developing the product and establishing its 
position in the marketplace are paid and it tends to be profitable. Firms will often try to use exten- 
sion Strategies (constructs). These are attempts to delay the decline stage of the product life cycle by 
introducing new versions of the product. In the decline stage, “products will consume a dispropor- 
tionate share of management time and financial resources relative to their potential future worth’”’ 
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(hypothesis). To make this hypothesis fully testable, we would need operational definitions for dis- 
proportionate share, time, resources, and future worth. 

The challenge for the researcher in this example is to build more comprehensive theories to explain 
and predict how modifying the product and other variables will benefit the firm. 


Models 


The term model is used in business research and other fields of business to represent phenomena 
through the use of analogy. A model is defined here as a representation of a system that is constructed 
to study some aspect of that system or the system as a whole. Models differ from theories in that a 
theory’s role is explanation whereas a model’s role is representation. 

Early models (and even those created as recently as the 1990s for mainframe computers) were enor- 
mously expensive and often incomprehensible to all but their developers. Modeling software, such as 
Excel, has made modeling more inexpensive and accessible. 

Models allow researchers and managers to characterize present or future conditions: the effect 
of advertising on consumer awareness or intention to purchase, a product distribution channel, 
brand switching behavior, an employee training program, and many other aspects of business. A 
model’s purpose is to increase our understanding, prediction, and control of the complexities of the 
environment. 

Exhibit 3-6 provides an example of a maximum-flow model used in management science. In this ex- 
ample, a European manufacturer of automobiles needs an increased flow of shipping to its Los Angeles 
distribution center to meet demand. However, the primary distribution channel is saturated and alterna- 
tives must be sought. Although this is a geographic model, more sophisticated network, mathematical, 
and path diagrams are subsequently created so that researchers can create hypotheses about the nature, 
relationship, and direction of causality among variables. 

Descriptive, predictive, and normative models are found in business research.’ Descriptive models 
are used frequently for more complex systems, such as the one in Exhibit 3-6. They allow visualiza- 
tion of numerous variables and relationships. Predictive models forecast future events (e.g., the Fourt 
and Woodlock model could be used to forecast basketball shoes for a market segment).? Normative 


>Exhibit 3-6 A Distribution Network Model 
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>Exhibit 3-7 The Role of Reasoning in Model Development 
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models are used chiefly for control, informing us about what actions should be taken. Models may also 
be static, representing a system at one point in time, or dynamic, representing the evolution of a system 
over time. 

Models are developed through the use of inductive and deductive reasoning, which we sug- 
gested previously is integral to accurate conclusions about business decisions. As illustrated 
in Exhibit 3-7, a model may originate from empirical observations about behavior based on 
researched facts and relationships among variables. Inductive reasoning allows the modeler to 
draw conclusions from the facts or evidence in planning the dynamics of the model. The modeler 
may also use existing theory, managerial experience, judgment, or facts deduced from known laws 
of nature. In this case, deductive reasoning serves to create particular conclusions derived from 
general premises. 

Models are an important means of advancing theories and aiding decision makers. Because the 
inputs are often unknown, imprecise, or temporal estimates of complex variables, creating and using 
models in the decision-making process can be a time-consuming endeavor. 


> Research and the Scientific Method 


Good business research is based on sound reasoning. Competent researchers and astute managers alike 
practice thinking habits that reflect sound reasoning—finding correct premises, testing the connec- 
tions between their facts and assumptions, making claims based on adequate evidence. In the reasoning 
process, induction and deduction, observation, and hypothesis testing can be combined in a systematic 
way. In this chapter we illustrate how this works and why careful reasoning is essential for producing 
scientific results. 

If the tools of thinking are the mind of science, then the scientific attitude is the spirit. The scientific 
attitude unleashes the creative drive that makes discovery possible. The portraits of scientists involved 
in some of the most spectacular discoveries of the last century—Crick, Watson, and Pauling (who 
developed the foundations of DNA structure), and others—are the stories of imagination, intuition, 
curiosity, suspicion, anguish, the rage to know, and self-doubt. A good business researcher must also 
possess these essential predispositions. Each must exercise imagination in the discovery process in cap- 
turing the most essential aspect of the problem, or in selecting a technique that reveals the phenomenon 
in its most natural state. 

Curiosity in its many forms characterizes the persistent effort to understand relationships. For ex- 
ample, consider the Hawthorne studies discovering a relationship between the seemingly unrelated 
entities of productivity and levels of light in the workplace. Exemplars like Weber, Taylor, Fayol, 
Gulick, Mayo, Maslow, McGregor, Argyris, Simon, Likert, Katz, and Kahn (among others in orga- 
nizational studies) have all typified the curiosity to ask questions with the passion not to quit and a 
discomfort with existing answers. From applied researchers addressing managers’ practical needs to 
academics fascinated with the construction of grand theories, the attitude of science is the enabling 
spirit of discovery. 
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Business and Battlefield: Scientific Evidence Supports “Gut-Hunches” 


A team of researchers at Leeds University Business School con- 
ducted studies on how intuition and hunches result from the 
way our brains store, process, and retrieve information on a 
subconscious level.* Their research is important to executives 
and managers, who often claim that hunches are preferred over 
deliberate analysis when a quick decision is required. Gerald 
Hodgkinson, the lead researcher remarked, “People usually ex- 
perience true intuition when they are under severe time pres- 
sure or in a situation of information overload or acute danger, 
where conscious analysis of the situation may be difficult or 
impossible.”° 

If we consider the analogy of the business executive attempt- 
ing to avoid fatal decisions in a fast-moving, turbulent environ- 
ment with a soldier scanning the landscape for any evidence of 
improvised explosive devices (IEDs) in a treacherous neighbor- 
hood, then the importance to organizations comes into focus. 
This aspect of decision-making research seeks to understand 
how to channel and fine-tune intuitive skills. Being able to identify 
when managers and executives switch from an intuitive mode to 
deliberate analysis, and why, may shed light on which decisions 
are likely to be correct for their environments. 

In an article on the importance of hunches in battle, Benedict 
Carey described how hunches were critical to military survival in 
threat environments like Iraq and Afghanistan, especially clearing 
roads of IEDs. Fighting a war-within-a-war, insurgents and U.S. 
bomb teams have improved their IED tactics: insurgents by bet- 
ter placing, concealing, and detonating and U.S. teams with bet- 
ter recognition and diffusion. Reduced casualties, it was found, 
could be attributed to some soldiers who could sense danger in 
a life-or-death situation well before others.° 

Army research psychologist Steven Burnett led a study in- 
volving approximately 800 military men and women that focused 
on how some soldiers see what others miss. They found that two 
types of personnel are particularly good at spotting anomalies: 
those with hunting backgrounds and those from tough urban 
neighborhoods, where knowing what gang controls a block is 
a survival necessity. These latter troops also seemed to have 
innate “threat-assessment” abilities.° 

Martin P. Paulus, a psychiatrist at the University of California, 
San Diego found that the brains of elite military units appear to 
record apparent threats differently from the average enlistee.® 
When presented with the sight of angry faces, Navy Seals show 
significantly higher activation in the insula (the brain location that 
collects sensations from around the body and interprets them 
cohesively) than regular soldiers. 


Not long ago, management academics thought of hunches 
and intuition as folklore, or as just feelings. Feelings have “little 
to do with rational decision making, or that got in the way of 
it,” said Dr. Antonio Damasio, director of the Brain and Creativ- 
ity Institute.’ Notwithstanding that hunches are still not part of 
scientific orthodoxy,? they are supported by strong evidence in 
neuroscience and psychology. 

The technical evidence is complicated but fascinating. Here 
is a glimpse. 


Each of us was born with two brains—the cranial brain 
between our ears, and a second brain with just as many 
neurons and neurotransmitters as the first, but located in 
the sheaths of tissue lining our stomach, small intestine, 
and colon. During early fetal development the same clump 
of embryonic tissue constituted both our primary brain and 
our gut brain. In later development the two brains sepa- 
rated yet remained connected (and in communication) 
through the vagus nerve extending from the brain stem 
through the enteric nervous system, otherwise known as 
our gut brain.” 


Receptors for the gut that process serotonin (a neurotrans- 
mitter) are identical to those found in the bilateral part of the 
brain where intuitive thinking is believed to originate. Professor 
Wolfgang Prinz of the Max Planck Institute in Munich reveals 
that our gut brain “may be the source for unconscious deci- 
sions which the main brain later claims as conscious decisions 
of its own.” 

There is a lot of research to support gut brain intuition; how- 
ever, a few examples are illustrative.’ At the Institute of Noetic 
Sciences, researchers showed how the human gut reacts to 
emotionally alarming information seconds before the conscious 
mind is aware of the information. Previous experiments found 
similar evidence of reaction times four to seven seconds before 
conscious awareness of emotionally disturbing images.‘ In an- 
other study by Professor Ronald Rensink of the University of 
British Columbia, one-third of the subjects could sense changes 
in the patterns of a series of images before the actual changes 
occurred. “It’s like a gut feeling,” said Rensink. “It’s like using the 
force. The point of this is that these kinds of feelings are often 
COnmecian 

Stephen Jay Gould, the eminent paleontologist, biologist, 
and historian of science said, “Science . . . progresses by hunch, 
vision, and intuition.”" Let’s not forget the hunches. 
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The scientific method, as practiced in business research, guides our approach to problem solving. 
The essential tenets of the scientific method are: 


e Direct observation of phenomena. 

¢ Clearly defined variables, methods, and procedures. 

e Empirically testable hypotheses. 

¢ The ability to rule out rival hypotheses. 

¢ Statistical rather than linguistic justification of conclusions. 


e The self-correcting process. 


An important term in this list is empirical. Empirical testing or empiricism is said “to denote obser- 
vations and propositions based on sensory experience and/or derived from such experience by methods 
of inductive logic, including mathematics and statistics.”'° Researchers using this approach attempt to 
describe, explain, and make predictions by relying on information gained through observation. This 
book is fundamentally concerned with empiricism—with the design of procedures to collect factual 
information about hypothesized relationships that can be used to decide if a particular understanding of 
a problem and its possible solution are correct. 

The scientific method, and scientific inquiry generally, is described as a puzzle-solving activity." 
For the researcher, puzzles are solvable problems that may be clarified or resolved through reasoning 
processes. The steps that follow represent one approach to assessing the validity of conclusions about 
observable events.'” They are particularly appropriate for business researchers whose conclusions re- 
sult from empirical data. The researcher: 


1. Encounters a curiosity, doubt, barrier, suspicion, or obstacle. 


2. Struggles to state the problem—asks questions, contemplates existing knowledge, gathers facts, 
and moves from an emotional to an intellectual confrontation with the problem. 


3. Proposes a hypothesis, a plausible explanation, to explain the facts that are believed to be 
logically related to the problem. 


4. Deduces outcomes or consequences of the hypothesis—attempts to discover what happens if the 
results are in the opposite direction of that predicted or if the results support the expectations. 
5. Formulates several rival hypotheses. 


6. Devises and conducts a crucial empirical test with various possible outcomes, each of which 
selectively excludes one or more hypotheses. 


7. Draws a conclusion (an inductive inference) based on acceptance or rejection of the hypotheses. 


8. Feeds information back into the original problem, modifying it according to the strength of the 
evidence. 


Clearly, reasoning is pivotal to much of the researcher’s success: gathering facts consistent with the 
problem, proposing and eliminating rival hypotheses, deducing outcomes, developing crucial empiri- 
cal tests, and deriving the conclusion. 


Sound Reasoning for Useful Answers 


Every day we reason with varying degrees of success and communicate our meaning in ordinary lan- 
guage or, in special cases, in symbolic, logical form. Our meanings are conveyed through one of two 
types of discourse: exposition or argument. Exposition consists of statements that describe without at- 
tempting to explain. Argument allows us to explain, interpret, defend, challenge, and explore meaning. 
Two types of argument of great importance to research are deduction and induction. 


Deduction 


Deduction is a form of argument that purports to be conclusive—the conclusion must necessarily fol- 
low from the reasons given. These reasons are said to imply the conclusion and represent a proof. This 
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Curiosity may have killed the cat—or 
at least gotten its paws wet, but it is 
an essential characteristic of a good 
researcher. Intelligent people who 
have a driving need to seek answers 
are at the heart of great research. 


is a much stronger and different bond between reasons and conclusions than is found with induction. 
For a deduction to be correct, it must be both true and valid: 


e Premises (reasons) given for the conclusion must agree with the real world (true). 
¢ The conclusion must necessarily follow from the premises (valid). 


A deduction is valid if it is impossible for the conclusion to be false if the premises are true. Logi- 
cians have established rules by which one can judge whether a deduction is valid. Conclusions are not 
logically justified if one or more premises are untrue or the argument form is invalid. A conclusion may 
still be a true statement, but for reasons other than those given. For example, consider the following 
simple deduction: 


All employees at BankChoice can be trusted to observe the ethical code. (Premise 1) 
Sara is an employee of BankChoice. (Premise 2) 
Sara can be trusted to observe the ethical code. (Conclusion) 


If we believe that Sara can be trusted, we might think this is a sound deduction. But this conclusion 
cannot be accepted as a sound deduction unless the form of the argument is valid and the premises 
are true. In this case, the form is valid, and premise 2 can be confirmed easily. However, more than a 
billion dollars each year in confirmed retail employee theft will challenge the premise “All employees 
can be trusted to observe an ethical code.” And instances of employee fraud among professionals make 
any specific instance questionable. If one premise fails the acceptance test, then the conclusion is not 
a sound deduction. This is so even if we still have great confidence in Sara’s honesty. Our conclusion, 
in this case, must be based on our confidence in Sara as an individual rather than on a general premise 
that all employees of BankChoice are ethical. 
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As researchers, we may not recognize how much we use deduction to understand the implications of 
various acts and conditions. For example, in planning a survey, we might reason as follows: 


Inner-city household interviewing is especially difficult and expensive. (Premise 1) 
This survey involves substantial inner-city household interviewing. (Premise 2) 
The interviewing in this survey will be especially difficult and expensive. (Conclusion) 


On reflection, it should be apparent that a conclusion that results from deduction is, in a sense, al- 
ready “contained in” its premises."° 


Induction 


Inductive argument is radically different. There is no such strength of relationship between reasons 
and conclusions in induction. In induction you draw a conclusion from one or more particular facts 
or pieces of evidence. The conclusion explains the facts, and the facts support the conclusion. To 
illustrate, suppose your firm spends $1 million on a regional promotional campaign and sales do not 
increase. This is a fact—sales did not increase during or after the promotional campaign. Under such 
circumstances, we ask, “Why didn’t sales increase?” 

One likely answer to this question is a conclusion that the promotional campaign was poorly ex- 
ecuted. This conclusion is an induction because we know from experience that regional sales should go 
up during a promotional event. Also we know from experience that if the promotion is poorly executed, 
sales will not increase. The nature of induction, however, is that the conclusion is only a hypothesis. It 
is one explanation, but there are others that fit the facts just as well. For example, each of the following 
hypotheses might explain why sales did not increase: 


¢ Regional retailers did not have sufficient stock to fill customer requests during the promotional 
period. 


° A strike by the employees of our trucking firm prevented stock from arriving in time for the 
promotion to be effective. 


e A category-five hurricane closed all our retail locations in the region for the 10 days during the 
promotion. 


In this example, we see the essential nature of inductive reasoning. The inductive conclusion is an 
inferential jump beyond the evidence presented—that is, although one conclusion explains the fact of 
no sales increase, other conclusions also can explain the fact. It may even be that none of the conclu- 
sions we advanced correctly explain the failure of sales to increase. 

For another example, let’s consider the situation of Tracy Nelson, a salesperson at the Square Box 
Company. Tracy has one of the poorest sales records in the company. Her unsatisfactory performance 
prompts us to ask the question “Why is she performing so poorly?” From our knowledge of Tracy’s 
sales practices, the nature of box selling, and the market, we might conclude (hypothesize) that her 
problem is that she makes too few sales calls per day to build a good sales record. Other hypotheses 
might also occur to us on the basis of available evidence. Among these hypotheses are the following: 


e Tracy’s territory does not have the market potential of other territories. 
e Tracy’s sales-generating skills are so poorly developed that she is not able to close sales effectively. 


e Tracy does not have authority to lower prices and her territory has been the scene of intense 
price-cutting by competitive manufacturers, causing her to lose many sales to competitors. 


e Some people just cannot sell boxes, and Tracy is one of those people. 


Each of the above hypotheses is an induction we might base on the evidence of Tracy’s poor 
sales record, plus some assumptions or beliefs we hold about her and the selling of boxes. All of 
them have some chance of being true, but we would probably have more confidence in some than in 
others. All require further confirmation before they gain our confidence. Confirmation comes with 
more evidence. The task of research is largely to (1) determine the nature of the evidence needed to 
confirm or reject hypotheses and (2) design methods by which to discover and measure this other 
evidence. 
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| Researchers often use 

observation when evaluating a 
customer's use of a product. 
Apply deductive reasoning to 
this image. Develop your own 
conclusions concerning what 
will happen next. 


Combining Induction and Deduction 


Induction and deduction are used together in research reasoning. Dewey describes this process as the 
“double movement of reflective thought.’’'* Induction occurs when we observe a fact and ask, “Why is 
this?” In answer to this question, we advance a tentative explanation (hypothesis). The hypothesis is 
plausible if it explains the event or condition (fact) that prompted the question. Deduction is the process 
by which we test whether the hypothesis is capable of explaining the fact. The process is illustrated in 
Exhibit 3-8: 


1. You promote a product but sales don’t increase. (Fact 1) 


2. You ask the question “Why didn’t sales increase?” (Induction) 


3. You infer a conclusion (hypothesis) to answer the question: The promotion was poorly 
executed. (Hypothesis) 


4. You use this hypothesis to conclude (deduce) that sales will not increase during a poorly 
executed promotion. You know from experience that ineffective promotion will not increase 
sales. (Deduction 1) 


This example, an exercise in circular reasoning, points out that one must be able to deduce the initiat- 
ing fact from the hypothesis advanced to explain that fact. A second critical point is also illustrated 
in Exhibit 3-8. To test a hypothesis, one must be able to deduce from it other facts that can then be 
investigated. This is what research is all about. We must deduce other specific facts or events from the 
hypothesis and then gather information to see if the deductions are true. In this example: 


5. We deduce that a well-executed promotion will result in increased sales. (Deduction 2) 

6. We run an effective promotion, and sales increase. (Fact 2) 
How would the double movement of reflective thought work when applied to Tracy Nelson’s problem? 
The process is illustrated in Exhibit 3-9. The initial observation (fact 1) leads to hypothesis 1 that 


Tracy is lazy. We deduce several other facts from the hypothesis. These are shown as fact 2 and fact 3. 
We use research to find out if fact 2 and fact 3 are true. If they are found to be true, they confirm our 
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> Exhibit 3-8 Why Didn’t Sales Increase? 


Induction 


Hypothesis: 
The promotion was 


poorly executed. 


Fact 2: 

We run an effective 
promotion and 
sales increase. 


> Exhibit 3-9 Why Is Tracy Nelson’s Performance So Poor? 


Induction 


Fact 2: Deduction: Laziness results 


Tracy is regularly Hypothesis: 


Tracy is lazy. 


late to work. in excessive tardiness. 


hypothesis. If they are found to be false, our hypothesis is not confirmed, and we must look for another 
explanation. 

In most research, the process may be more complicated than these examples suggest. For instance, 
we often develop multiple hypotheses by which to explain the problem in question. Then we design 
a study to test all the hypotheses at once. Not only is this more efficient, but it is also a good way to 
reduce the attachment (and potential bias) of the researcher for any given hypothesis. 


1. Scientific methods and scientific thinking are based on to think about and communicate abstractions. We also 
concepts, the symbols we attach to bundles of meaning use higher-level concepts— constructs —for special- 
that we hold and share with others. We invent concepts ized scientific explanatory purposes that are not directly 


observable. Concepts, constructs, and variables may be 
defined descriptively or operationally. Operational defini- 
tions must specify adequately the empirical information 
needed and how it will be collected. In addition, they must 
have the proper scope or fit for the research problem at 
hand. 

Concepts and constructs are used at the theoretical lev- 
els; variables are used at the empirical level. Variables accept 
numerals or values for the purpose of testing and measure- 
ment. They may be classified as explanatory, independent, 
dependent, moderating, extraneous, and intervening. 


2 Propositions are of great interest in research because 
they may be used to assess the truth or falsity of relation- 
ships among observable phenomena. When we advance a 
proposition for testing, we are hypothesizing. A hypothesis 
describes the relationships between or among variables. 
A good hypothesis is one that can explain what it claims 
to explain; is testable; and has greater range, probability, 
and simplicity than its rivals. Sets of interrelated concepts, 
definitions, and propositions that are advanced to explain 


>chapter 3 Thinking Like a Researcher 71 


and predict phenomena are called theories. Models differ 
from theories in that models are analogies or representa- 
tions of some aspect of a system or of the system as a 
whole. Models are used for description, prediction, and 
control. 


Scientific inquiry is grounded in the inference process. This 
process is used for the development and testing of vari- 
ous propositions largely through the double movement of 
reflective thinking. Reflective thinking consists of sequencing 
induction and deduction in order to explain inductively (by 
hypothesis) a puzzling condition. In turn, the hypothesis is 
used in a deduction of further facts that can be sought to 
confirm or deny the truth of the hypothesis. 

Researchers think of the doing of science as an orderly 
process that combines induction, deduction, observation, 
and hypothesis testing into a set of reflective thinking ac- 
tivities. Although the scientific method consists of neither 
sequential nor independent stages, the problem-solving pro- 
cess that it reveals provides insight into the way research is 
conducted. 


>k rm 
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iscussi | 


Terms in Review 
1. Distinguish among the following sets of items, and suggest 
the significance of each in a research context: 


a Concept and construct. 
b Deduction and induction. 
c Operational definition and dictionary definition. 


d Concept and variable. 
e Hypothesis and proposition. 
f Theory and model. 
g Scientific method and scientific attitude. 
2 Describe the characteristics of the scientific method. 


3 Below are some terms commonly found in a management 
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setting. Are they concepts or constructs? Give 
two different operational definitions for each. 


First-line supervisor. 


Employee morale. 
Assembly line. 

Overdue account. 
Line management. 


Leadership. 


QqgQsoaQagQoae»or 


Union democracy. 


h_ Ethical standards. 


In your company’s management development pro- 
gram, there was a heated discussion between some 
people who claimed, “Theory is impractical and thus 
no good,” and others who claimed, “Good theory 

is the most practical approach to problems.” What 
position would you take and why? 


5 An automobile manufacturer observes the de- 


mand for its brand increasing as per capita in- 

come increases. Sales increases also follow low 

interest rates, which ease credit conditions. Buyer 
purchase behavior is seen to be dependent on 
age and gender. Other factors influencing sales 
appear to fluctuate almost randomly (competitor 
advertising, competitor dealer discounts, intro- 
ductions of new competitive models). 

a lf sales and per capita income are positively 
related, classify all variables as dependent, 
independent, moderating, extraneous, or 
intervening. 


b Comment on the utility of a model based on the 


tardiness. Your inclination is to urge this 20- 
year employee to retire or to fire him, if retire- 
ment is rejected, but you know the individual is 
well liked and seen as a fixture in the company. 


a Pose several hypotheses that might account 
for dropped or misrouted incoming calls. 


b Using the double movement of reflective 
thought, show how you would test these 
hypotheses. 


Bringing Research to Life 


8 Identify and classify all the variables in the Army’s 
dud shell research. 


9 What was Myra’s hypothesis for the Army’s dud 
shell research? What was the Army’s hypothesis? 


From Concept to Practice 
10 Using Exhibits 3-1 and 3-9 as your guides, graph 


the inductions and deductions in the following 
statements. If there are gaps, supply what is 
needed to make them complete arguments. 


a Repeated studies indicate that economic 
conditions vary with—and lag 6 to 12 months 
behind—the changes in the national money 
supply. Therefore, we may conclude the money 
supply is the basic economic variable. 


b Research studies show that heavy smokers 
have a higher rate of lung cancer than do 
nonsmokers; therefore, heavy smoking causes 
lung cancer. 


c Show me aperson who goes to church regu- 
larly, and | will show you a reliable worker. 


From the Headlines 
11 Everywhere you look, there’s compelling evidence 
that the single-minded pursuit of wealth runs con- 


hypothesis. 


Making Research Decisions 
6 You observe the following condition: “Our female 


sales representatives have lower customer defec- 
tions than do our male sales representatives.” 


a Propose the concepts and constructs you 
might use to study this phenomenon. 


b How might any of these concepts and/or con- 
structs be related to explanatory hypotheses? 


You are the office manager of a large firm. Your 
company prides itself on its high-quality cus- 
tomer service. Lately complaints have surfaced 
that an increased number of incoming calls are 
being misrouted or dropped. Yesterday, when 
passing by the main reception area, you no- 
ticed the receptionist fiddling with his hearing 
aid. In the process, a call came in and would 
have gone unanswered if not for your interven- 
tion. This particular receptionist had earned an 
unsatisfactory review three months earlier for 


trary to the scientific attitude. A big story touching 
the investment banking community was the con- 
viction of Raj Rajaratnam on 14 counts of insider 
trading. Rajaratnam’s greed-driven scheme, ac- 
cording to the Harvard Business Review, will lead 
to destroyed reputations and significant prison 
terms for senior officials at IBM, McKinsey, and 
other companies. 


Before that, the big story was the resignation 
and disgrace of Berkshire Hathaway’s David 
Sokol, who was once thought to be the likely 
successor to CEO Warren Buffet. Sokol’s zeal 
to cash in suspiciously timed investments of the 
stock of a company Berkshire later bought, led to 
his humiliation. How does one reconcile the sci- 
entific attitude and role of “manager as scientist” 
with a willingness to risk everything to achieve 
more money than they'll ever need? 
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from the Online Learning Center. 


chapter 4 


learningobjectives 


After reading this chapter, you should understand. . . 


Research is decision- and dilemma-centered. 


The clarified research question is the result of careful exploration and analysis and sets the direction for the research 
project. 


How value assessments and budgeting influence the process for proposing research and, ultimately, research 
design. 


What is included in research design, data collection, data analysis, and reporting. 


Research process problems to avoid. 


C6 


Listening is the front end of decision 
making. It’s the surest, most efficient route 
to informing the judgments we need to 


make. 9 9 


Bernard Ferrari, consultant, Ferrari Consultancy, LLC, 
and former director at McKinsey & Company 


bringingresearchtolife...£7 


On the return flight from Austin, Jason Henry and 
MindWriter’s Myra Wines are discussing their trip. 
“That went really well,” she says. 

“There are going to be a few problems,” disagrees 
Jason. 

“Gracie wants the sun, the sky, and the moon. She, 
like most managers, wants to know the demographic 
characteristics of her users . . . their job descriptions . . . 
their salaries . . . their ethnicities . . . their education. 
Wants to know the perception of MindWriter; wants to 
know their satisfaction with the purchase channel and 
with CompleteCare service, too.” 

“And your point is?” asks Myra. 

“You and Gracie need to keep your eye on the bottom 
line. You can bet someone will want to know how you 
and Gracie can justify asking all these questions. They 
will ask, “What is going to be the payoff in knowing 
the ethnicity of customers?’ And if you or Gracie can’t 
explain the justification for needing the information, 
if one of you can’t establish that the dollar benefit of 
knowing is at least as great as the dollar cost of finding 
out, the question will get struck from the developing 
research.” 

“Ts there no way we can justify knowing everything 
Gracie wants to know?” inquires Myra. 

“We can do a pilot study by survey of a few hundred 
customers and see if the ethnic background, or the 
salary level, or any other item that Gracie cares about is 
a good indicator of satisfaction, willingness to make a 
repeat purchase, postpurchase service satisfaction, and 
so forth. If it is, maybe more extensive measurement 
can be justified.” 

“So you feel we need to propose an exploratory 
study to whittle down the information to critical items, 
followed by a larger study.” 


“A pilot study could help in other ways, too. 
Gracie wants to know the customers’ perceptions 
of MindWriter’s overall quality. But we have to ask 
ourselves, “Are these customers really qualified to form 
independent opinions, or will they simply be parroting 
what they have read in the computer magazines or what 
a dealer told them?’ A pilot study of a few hundred 
users can help determine if it is really useful to ask 
them their overall impression of the product. 

“However, with the repair problem, we can be 
reasonably sure that CompleteCare customers know 
their own minds when it comes to evaluating their 
firsthand experience with MindWriter’s service 
department.” 

“Today’s tour of the CompleteCare facility really 
helped me understand the context of management’s 
concern,” comments Myra. “Did you or Sara have a 
chance to look over any of the customer letters from 
the service department?” 

Jason digs into his briefcase and extracts a 
small sheaf of photocopies. “Yes, and Sara had 
reviewed transcriptions, too, on service center 
phone conversations. She pulled a few for us. One 
person writes, ‘My MindWriter was badly damaged 
on arrival. I could not believe its condition when I 
unpacked it.’ And here, ‘The service technicians 
seemed to be unable to understand my complaint, but 
once they understood it, they performed immediate 
repairs.’ You and I will collaborate to boil down 
these, and possibly dozens more like them, to a couple 
of representative questions that can be pilot-tested 
for clarity, consistency, and representativeness. You 
don’t want MindWriter to pay for everything Gracie 
says she wants, just what she wants that has a payoff 
and is researchable.” 
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> The Research Process 


Writers usually treat the research task as a sequential process involving several clearly defined steps. 
No one claims that research requires completion of each step before going to the next. Recycling, 
circumventing, and skipping occur. Some steps are begun out of sequence, some are carried out 
simultaneously, and some may be omitted. Despite these variations, the idea of a sequence is useful for 
developing a project and for keeping the project orderly as it unfolds. 

Exhibit 4-1 models the sequence of the research process. We refer to it often as we discuss each step 
in subsequent chapters. Our discussion of the questions that guide project planning and data gathering 


> Exhibit 4-1 The Research Process 


Chapters 2-5 


Exploration Exploration 


Appendix A 
Research Design 
Strategy 
(type, purpose, time frame, scope, environment) Chapters 6-14 
Data Collection Sampling 
Design Design 
Data Collection & Preparation Chapter 15 


Chapters 16-18 


Research Reporting 
Chapters 19-20 


~<—Stage 6>~<¢ Stage 5-»< Stage 4—» <—-Stage 3——__» =< Stage 2—__» <= Stage 1 


Management 
Decision 


>chapter 4 The Research Process: An Overview 


is incorporated into the model (see the elements within the pyramid in Exhibit 4-1). Exhibit 4-1 also 
organizes this chapter and introduces the remainder of the book. 

The research process begins much as the vignette suggests. A management dilemma triggers 
the need for a decision. For MindWriter, a growing number of complaints about postpurchase ser- 
vice started the process. In other situations, a controversy arises, a major commitment of resources 
is called for, or conditions in the environment signal the need for a decision. For MindWriter, 
the critical event could have been the introduction by a competitor of new technology that would 
revolutionize the processing speed of laptops. Such events cause managers to reconsider their 
purposes or objectives, define a problem for solution, or develop strategies for solutions they have 
identified. 

In our view of the research process, the management question—its origin, selection, statement, 
exploration, and refinement—is the critical activity in the sequence. Throughout the chapter we em- 
phasize problem-related steps. A familiar quotation from Albert Einstein, no less apt today than when 
it was written, supports this view: 


The formulation of a problem is far more often essential than its solution, which may be merely a matter of mathemati- 
cal or experimental skill. To raise new questions, new possibilities, to regard old problems from a new angle requires 
creative imagination and marks real advance in science.' 


Whether the researcher is involved in basic or applied research, a thorough understanding of the man- 
agement question is fundamental to success in the research enterprise. 


> Stage 1: Clarifying the Research Question 


A useful way to approach the research process is to state the basic dilemma that prompts the research 
and then try to develop other questions by progressively breaking down the original question into more 
specific ones. You can think of the outcome of this process as the management-research question 
hierarchy. Exhibit 4-2 follows the MindWriter example through the process. 

The process begins at the most general level with the management dilemma. This is usually a 
symptom of an actual problem, such as: 


¢ Rising costs. 


¢ The discovery of an expensive chemical compound that would increase the efficacy of 
a drug. 


e Increasing tenant move-outs from an apartment complex. 

¢ Declining sales. 

¢ Increasing employee turnover in a restaurant. 

e A larger number of product defects during the manufacture of an automobile. 


e An increasing number of letters and phone complaints about postpurchase service (as in 
MindWriter; see Exhibit 4-2). 


The management dilemma can also be triggered by an early signal of an opportunity or grow- 
ing evidence that a fad may be gaining staying power—like the growing interest in hybrid cars— 
indicated by the number of broadcast news segments and print stories over an extended period of 
time. 

Identifying management dilemmas is rarely difficult (unless the organization fails to track its per- 
formance factors—like sales, profits, employee turnover, manufacturing output and defects, on-time 
deliveries, customer satisfaction, etc.). However, choosing one dilemma on which to focus may be 
difficult. Choosing incorrectly will direct valuable resources (time, manpower, money, and equipment) 
on a path that may not provide critical decision-making information (the purpose of good research). As 
a manager, only practice makes you proficient. For new managers, or established managers facing new 
responsibilities, developing several management-research question hierarchies, each starting with a 
different dilemma, will assist in the choice process. In all figures related to the research process model, 
in this and subsequent chapters, we use an inverted pyramid to represent the management-research 
question hierarchy. 
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>Exhibit 4-2 Formulating the Research Question for MindWriter 

To move from the management dilemma to the management question and subsequent research questions takes exploratory research. 
Such research may include examining previous studies, reviewing published studies and organizational records, and interviewing experts 
or information gatekeepers. 


ta Exploration 


An increasing number of letters 


; Stage 1: Pre-Austin Stage 3: Post-Austin: 
and phone complaints about 1. PC magazines: annual survey Brainstorming & company 
postpurchase service. of service, repair, & tech letters 
support 1. Possible problems: 
2. Published customer (a) Employee shortages 
satisfaction comparisons (b) Tech-line operator training 
Stage 2: Austin Meeting (c) Uneven courier performance 
1. Production: 5,000/mo. (d) Parts shortages 
2. Distribution through computer (e) Inconsistent repair servicing 
superstores and independent (f) Product damage during repair 
mail order co. (g) Product damage during 


. CustomCare process shipping 
(h) Packaging and handling 


problems 


What should be done to improve 
the CompleteCare program for 
MindWriter product repairs 
and servicing? 


2a Exploration 
Interviews with 
© Service manager 
¢ Call center manager 
e Independent package 
company account 
executive 


© Should the tech-support operator be 
given more intensive training? 

e Should ABC Courier Service be 
replaced by an air-transport service? 

e Should the repair diagnostic and 


repair sequencing operations be 
modified? 


e Should the return packaging be 
modified to include premolded rigid 
foam inserts, or conforming-expanding 
foam protection? 

e Should metropolitan repair centers 
be established to complement 
or replace in-factory repair facilities? 
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The Container Store teams with OpinionLab to gain feedback 


The Container Store is a favorite among highly organized people 
and those wishing to be. In addition to its stores, its website 
offers a wealth of ideas as well as access to manufacturer- and 
private-brand inventory. A customer can enter data that permits 
The Container Store personnel to recommend an elfa® closet 
design or watch videos to enable them to do storage projects 
themselves. Lucy Witte, the company’s vice president of direct 
marketing & media, is charged with creating the best website 
experience for customers. When the website is updated, she 
wants to identify potential technical problems that might not be 
caught during pre-testing. 

The Container Store partnered with OpinionLab to use its 
feedback platform to involve customers in making the website as 
powerful as it needs to be. OpinionLab provides feedback from 
any PC or mobile device so a research respondent can share 
direct input on a product or website feature. Each page on the 
website features a [+] feedback link—a revolving +/— sign—in 
the lower right corner. When a customer clicks on the link, a 
comment window appears. “OpinionLab allows us to identify 
and resolve issues as they emerge and this is a tremendous 
tool for improving our customers’ experience across channels, 
which is increasingly important as our customers turn to tablets, 
mobile, stores and desktops to shop,” shared Witte. Opinion- 
Lab’s platform makes it much easier for customers to interact 
with the company in real time, the new standard for 21st century 
research. 

The Container Store received more than it initially intended 
with its OpinionLab feedback process. Besides learning, for 
example, that photos weren’t clear, customers also asked 
questions about which products would best fit their needs 
and commented positively and negatively on their in-store 


The Original Storage and Organization Store? 
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experiences. “We share these comments with our store manag- 
ers and staff,” explained Witte, indicating that the feedback has 
influenced in-store staff training. 

According to Jonathan Levitt, chief marketing officer of 
OpinionLab, “By understanding shoppers’ motivations and rea- 
sons for [online purchase] abandonment, The Container Store 
was able to quantify [and understand] multichannel purchase 
behaviors and optimize the experience for their best sellers.” 


www.containerstore.com; www.opinionlab.com 


Subsequent stages of the hierarchy take the manager and his or her research collaborator through 


various brainstorming and exploratory research exercises to define the following: 


e Management question—a restatement of the manager’s dilemma(s) in question form. 


e Research questions—the hypothesis that best states the objective of the research; the 


question(s) that focuses the researcher’s attention. 


¢ Investigative questions—questions the researcher must answer to satisfactorily answer the 
research question; what the manager feels he or she needs to know to arrive at a conclusion 


about the management dilemma. 


e Measurement questions—what participants in research are asked or what specifically is 


observed in a research study. 


The definition of the management question sets the research task. A poorly defined management 


question will misdirect research efforts. In Chapter 5 we explore this critical stage in more detail in our 
search to clarify the research question. 
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> Stage 2: Proposing Research 


Resource Allocation and Budgets 


General notions about research budgets have a tendency to single out data collection as the most costly 
activity. Data collection requires substantial resources but perhaps less of the budget than clients expect. 
Employees must be paid, training and travel must be provided, and other expenses incurred must be 
paid; but this phase of the project often takes no more than one-third of the total research budget. The 
geographic scope and the number of observations required do affect the cost, but much of the cost is 
relatively independent of the size of the data-gathering effort. Thus, a guide might be that (1) project 
planning; (2) data gathering; and (3) analysis, interpretation, and reporting each shares about equally 
in the budget. 

Without budgetary approval, many research efforts are terminated for lack of resources (see Ex- 
hibit 4-3). A budget may require significant development and documentation as in grant and contract 
research, or it may require less attention as in some in-house projects or investigations funded out of the 
researcher’s own resources. The researcher who seeks funding must be able not only to persuasively jus- 
tify the costs of the project but also to identify the sources and methods of funding. One author identifies 
three types of budgets in organizations where research is purchased and cost containment is crucial: 


° Rule-of-thumb budgeting involves taking a fixed percentage of some criterion. For example, a 
percentage of the prior year’s sales revenues may be the basis for determining the marketing 
research budget for a manufacturer. 


¢ Departmental or functional area budgeting allocates a portion of total expenditures in the unit 
to research activities. Government agencies, not-for-profits, and the private sector alike will 
frequently manage research activities out of functional budgets. Units such as human resources, 
marketing, or engineering then have the authority to approve their own projects. 


° Task budgeting selects specific research projects to support on an ad hoc basis. This type is the 
least proactive but does permit definitive cost-benefit analysis.” 


> Exhibit 4-3 Proposing Research 
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Valuing Research Information 


There is a great deal of interplay between budgeting and value assessment in any management decision 
to conduct research. An appropriate research study should help managers avoid losses and increase 
sales or profits; otherwise, research can be wasteful. The decision maker wants a firm cost estimate for 
a project and an equally precise assurance that useful information will result from the study. Even if 
the researcher can give good cost and information estimates, the managers still must judge whether the 
benefits outweigh the costs. 

Conceptually, the value of applied research is not difficult to determine. In a business situation, the 
research should produce added revenues or reduce expenses in much the same way as any other invest- 
ment of resources. One source suggests that the value of research information may be judged in terms 
of “the difference between the result of decisions made with the information and the result that would 
be made without it.”* While such a criterion is simple to state, its actual application presents difficult 
measurement problems. 


Evaluation Methods 


Ex Post Facto Evaluation If there is any measurement of the value of research, it is usually an 
after-the-fact event. Twedt reports on one such effort, an evaluation of marketing research done at a 
major corporation.* He secured “an objective estimate of the contribution of each project to corporate 
profitability.” He reports that most studies were intended to help management determine which one of 
two (or more) alternatives was preferable. He guesses that in 60 percent of the decision situations, the 
correct decision would have been made without the benefit of the research information. In the remain- 
ing 40 percent of the cases, the research led to the correct decision. Using these data, he estimates that 
the return on investment in marketing research in this company was 3.5 times for the year studied. 
However, he acknowledges the return-on-investment figure was inflated because only the direct re- 
search costs were included. 

This effort at cost-benefit analysis is commendable even though the results come too late to guide a 
current research decision. Such analysis may sharpen the manager’s ability to make judgments about 
future research proposals. However, the critical problem remains, that of project evaluation before the 
study is done. 


Prior or Interim Evaluation A proposal to conduct a thorough management audit of operations 
in a company may be a worthy one, but neither its costs nor its benefits are easily estimated in advance. 
Such projects are sufficiently unique that managerial experience seldom provides much aid in evaluat- 
ing such a proposal. But even in these situations, managers can make some useful judgments. They may 
determine that a management audit is needed because the company is in dire straits and management 
does not understand the scope of its problems. The management information need may be so great as 
to ensure that the research is approved. In such cases, managers may decide to control the research 
expenditure risk by doing a study in stages. They can then review costs and benefits at the end of each 
stage and give or withhold further authorization. 


Option Analysis Some progress has been made in the development of methods for assessing the 
value of research when management has a choice between well-defined options. Managers can conduct 
a formal analysis with each alternative judged in terms of estimated costs and associated benefits and 
with managerial judgment playing a major role. 

If the research design can be stated clearly, one can estimate an approximate cost. The critical task 
is to quantify the benefits from the research. At best, estimates of benefits are crude and largely reflect 
an orderly way to estimate outcomes under uncertain conditions. To illustrate how the contribution of 
research is evaluated in such a decision situation, we must digress briefly into the rudiments of decision 
theory. 


Decision Theory When there are alternatives from which to choose, a rational way to approach 
the decision is to try to assess the outcomes of each action. The case of two choices will be discussed 
here, although the same approach can be used with more than two choices. 
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Two possible actions (A, and A,) may represent two different ways to organize a company, pro- 
vide financing, produce a product, and so forth. The manager chooses the action that affords the 
best outcome—the action choice that meets or exceeds whatever criteria are established for judging 
alternatives. Each criterion is a combination of a decision rule and a decision variable. The deci- 
sion variable might be “direct dollar savings,” “contribution to overhead and profits,” “time required 
for completion of the project,” and so forth. For MindWriter, the decision variable might be number 
of postservice complaints or the level of postservice satisfaction. Usually the decision variable is 
expressed in dollars, representing sales, costs, some form of profits or contribution, or some other 
quantifiable measure. The decision rule may be “choose the course of action with the lowest loss 
possibility” or perhaps “choose the alternative that provides the greatest annual net profit.” For 
MindWriter, the decision rule might be “choose the alternative that provides the highest level of 
postservice satisfaction.” 

The alternative selected (A, versus A,) depends on the decision variable chosen and the decision 
rule used. The evaluation of alternatives requires that (1) each alternative is explicitly stated, (2) a deci- 
sion variable is defined by an outcome that may be measured, and (3) a decision rule is determined by 
which outcomes may be compared. 


29 66. 


The Research Proposal 


Exhibit 4-1 depicts the research proposal as an activity that incorporates decisions made during early 
project planning phases of the study, including the management-—research question hierarchy and ex- 
ploration. The proposal thus incorporates the choices the investigator makes in the preliminary steps, 
as depicted in Exhibit 4-3. 

A written proposal is often required when a study is being suggested. This is especially true if an 
outside research supplier will be contracted to conduct the research. The written proposal ensures 
that the parties concur on the project’s purpose, the proposed methods of investigation, the extent of 
analysis, and the timing of each phase as well as of delivery of results. Budgets are spelled out, as 
are other responsibilities and obligations. The proposal may serve the purpose of a legally binding 
contract. 

A research proposal also may be oral, wherein all aspects of the research are discussed but not 
codified in writing. This is more likely when a manager directs his or her own research or the research 
activities of subordinates. We describe detailed research proposals in Appendix A, and you will find a 
sample proposal on the text website. 


> Stage 3: Designing the Research Project 


Research Design 


The research design is the blueprint for fulfilling objectives and answering questions. Selecting a 
design may be complicated by the availability of a large variety of methods, techniques, procedures, 
protocols, and sampling plans. For example, you may decide on a secondary data study, case study, 
survey, experiment, or simulation. If a survey is selected, should it be administered by mail, computer, 
telephone, the Internet, or personal interview? Should all relevant data be collected at one time or at 
regular intervals? What kind of structure will the questionnaire or interview guide possess? What ques- 
tion wording should be employed? Should the responses be scaled or open-ended? How will reliability 
and validity be achieved? Will characteristics of the interviewer influence responses to the measure- 
ment questions? What kind of training should the data collectors receive? Is a sample or a census to 
be taken? What types of sampling should be considered? These questions represent only a few of the 
decisions that have to be made when just one method is chosen. 

Although selecting an appropriate design may be complicated by this range of options, the creative 
researcher actually benefits from this confusing array of options. The numerous combinations spawned 
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Emerging Research Techniques 
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According to the 2012 Greenbook Research Industry Trends (GRIT) report, the top four emerging techniques, among both 
research buyers and providers all involve Internet use. “A big climber, from actual 2011 to expected 2012, is Mobile Surveys, 
with clients/buyers jumping from a current 17% to an expected 53% and vendors expecting the increase to be from 24% to 
64%.” Some speculate that the mobile survey may be approaching its tipping point. Other methodologies, like Mobile Qualita- 
tive, Mobile Ethnography, and Gamification, are getting a lot of buzz in the industry, but have yet to capture buyer/client sup- 
port to the same degree that they have earned researcher interest. As in previous studies, researcher interest tends to lead on 


methodology. http://www.greenbook.org/PDFs/GRIT-S12-Full.pdf 


Source: “Spring 2012 Greenbook Research Trends Report,” GreenBook® | New York AMA Communication Services Inc., February 2012, p. 22. 


Leonard Murphy, “GRIT Sneak Peek: What Emerging Research Techniques Will Be Used in 2012?” Greenbook, posted February 20, 2012. 
Downloaded April 18, 2012, http://www.greenbookblog.org/201 2/02/20/grit-sneak-peek-what-emerging-research-techniques-will-be- 


used-in-201 2/. 


by the abundance of tools may be used to construct alternative perspectives on the same problem. By 
creating a design using diverse methodologies, researchers are able to achieve greater insight than 
if they followed the most frequently used method or the method receiving the most media attention. 
Although pursuing research on a single research problem from a multimethod, multistudy strategy is 
not currently the norm, such designs are getting increasing attention from researchers and winning 
numerous industry awards for effectiveness. The advantages of several competing designs should be 
considered before settling on a final one. 

Jason’s preference for MindWriter is to collect as much information as possible from an explora- 
tion of company records, interviews with company managers of various departments, and multiple 
phone surveys with CompleteCare service program users. Financial constraints, however, might force 
MindWriter to substitute a less expensive methodology: a self-administered study in the form of an 
email-recruited, online survey using a sample drawn from the CompleteCare program users followed 
by phone contact with those who don’t participate. 
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New Moms and Moms-to-Be Offer Video Insights 


Doing research with mothers of infants and small children or ex- 
pectant mothers can be problematic. These women between 
21-40 have lots of distractions and their time availability is often 
at odd hours. Marcus Thomas, LLC (they describe themselves 
as an idea agency) and QualVu (a company that helps their cli- 
ents find truth) collaborated to help a durable baby goods pro- 
ducer better understand a mother’s choices. 

Specifically, the study objectives were to “gain insight into the 
mother’s perceptions of specific products, uncover their path 
to purchase, [understand] purchase motivations, and [share 
their] experiences at key retailers (Target, Walmart, and Babies 
‘R’ Us),” explained Jennifer Hirt-Marchand, associate partner, 
strategic insights executive with Marcus Thomas. Qualitative 
research would allow Marcus Thomas a deeper understanding 
of their subjects, so they opted for video diaries and QualVU 
offered the platform. 

Recruited participants needed to have a baby or be preg- 
nant, and have access to a computer and high-speed Internet 
access. Those that didn’t already have a video camera were 
provided one (that they could keep at the end of the study). “We 
knew from secondary research provided by Forrester Research 
that women in the age group were active on social networks 
and accustomed to sharing their personal stories online, using 
both text and video,” shared Edwige Winans, associate research 
director at Marcus Thomas. As moderator, Winans pre-recorded 
“discussion points” (pictured) and posted them to the QualVu 
platform, where participants accessed them. “What we tried to 
achieve was a semblance of a relationship with the participant, 
using the video.” The participants answered various questions 
by uploading video diaries to the site over a four-day period. The 
participants actually have “conversations with themselves,” ex- 
plained Winans. “Their diaries provide really candid responses.” 
Participants answer different discussion topics each day; the 
methodology allows participants to answer the questions on 
their own schedule. Videos are monitored while the study is 


Sampling Design 


ongoing, and the moderator uses follow-up emails to prompt or 
redirect participants for further clarification, if necessary. 

Over the course of the study 21 compensated participants 
responded to 8 topics with uploaded diaries on each topic (a 
typical video diary study is 7 to 10 topics). QualVu research- 
ers watched the videos, identifying trends in comments and 
provided a video report. “The client was especially interested in 
comments on retailer in-store merchandising elements,” shared 
Winans. “But our most important task was to keep an open 
mind.” Some of the most valuable insights came from secondary 
cues, like the participant's home environment, props the partici- 
pant used in her diaries, and the product demonstrations she did 
in stores or in her home. 

“A key consideration in recommending this methodology to 
our client was to ensure that technology was intuitive and natural 
to the participant and allow them to comfortably and confidently 
share their thoughts and perspectives,” stated Hirt-Marchand. 
The client got just what they needed. 


www.marcusthomasllc.com; www.qualvu.com 


Another step in planning the research project is to identify the target population (those people, events, 
or records that contain the desired information and can answer the measurement questions) and then 
determine whether a sample or a census is desired. Taking a census requires that the researcher examine 
or count all elements in the target population. A sample examines a portion of the target population, and 
the portion must be carefully selected to represent that population. If sampling is chosen, the researcher 
must determine which and how many people to interview, which and how many events to observe, or 
which and how many records to inspect. When researchers undertake sampling studies, they are inter- 
ested in estimating one or more population values (such as the percent of satisfied service customers 
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who will buy new MindWriter laptops when the need arises) and/or testing one or more statistical 
hypotheses (e.g., that highly satisfied CompleteCare service customers will be far more likely to repur- 
chase the MindWriter brand of laptops). 

If a study’s objective is to examine the attitudes of U.S. automobile assemblers about quality im- 
provement, the population may be defined as the entire adult population of auto assemblers employed 
by the auto industry in the United States. Definition of the terms adult and assembler and the relevant 
job descriptions included under “assembly” and “auto industry” may further limit the population under 
study. The investigator may also want to restrict the research to readily identifiable companies in the 
market, vehicle types, or assembly processes. 

The sampling process must then give every person within the target population a known nonzero 
chance of selection if probability sampling is used. If there is no feasible alternative, a nonprobability 
approach may be used. Jason knows that his target population comprises MindWriter customers who 
have firsthand experience with the CompleteCare program. Given that a list of CompleteCare program 
users emails (a sample frame) is readily available each month, a probability sample is feasible. 


Pilot Testing 


The data-gathering phase of the research process typically begins with pilot testing. Pilot testing may 
be skipped when the researcher tries to condense the project time frame. 

A pilot test is conducted to detect weaknesses in design and instrumentation and to provide proxy 
data for selection of a probability sample. It should, therefore, draw subjects from the target population 
and simulate the procedures and protocols that have been designated for data collection. If the study is 
a survey to be executed by mail, the pilot questionnaire should be mailed. If the design calls for obser- 
vation by an unobtrusive researcher, this behavior should be practiced. The size of the pilot group may 
range from 25 to 100 subjects, depending on the method to be tested, but the respondents do not have 
to be statistically selected. In very small populations or special applications, pilot testing runs the risk 
of exhausting the supply of respondents and sensitizing them to the purpose of the study. This risk is 
generally overshadowed by the improvements made to the design by a trial run. 

There are a number of variations on pilot testing. Some of them are intentionally restricted to data 
collection activities. One form, pretesting, may rely on colleagues, respondent surrogates, or actual 
respondents to refine a measuring instrument. This important activity has saved countless survey stud- 
ies from disaster by using the suggestions of the respondents to identify and change confusing, awk- 
ward, or offensive questions and techniques. One interview study was designed by a group of college 
professors for EducTV, an educational television consortium. In the pilot test, they discovered that 
the wording of nearly two-thirds of the questions was unintelligible to the target group, later found to 
have a median eighth-grade education. The revised instrument used the respondents’ language and was 
successful. Pretesting may be repeated several times to refine questions, instruments, or procedures. 


> Stage 4: Data Collection and Preparation 


The gathering of data may range from a simple observation at one location to a grandiose survey of 
multinational corporations at sites in different parts of the world. The method selected will largely de- 
termine how the data are collected. Questionnaires, standardized tests, observational forms, laboratory 
notes, and instrument calibration logs are among the devices used to record raw data. 

But what are data? One writer defines data as the facts presented to the researcher from the study’s 
environment. First, data may be further characterized by their abstractness, verifiability, elusiveness, 
and closeness to the phenomenon.° As abstractions, data are more metaphorical than real. For example, 
the growth in GDP cannot be observed directly; only the effects of it may be recorded. Second, data are 
processed by our senses—often limited in comparison to the senses of other living organisms. When 
sensory experiences consistently produce the same result, our data are said to be trustworthy because 
they may be verified. Third, capturing data is elusive, complicated by the speed at which events occur 
and the time-bound nature of observation. Opinions, preferences, and attitudes vary from one milieu 
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to another and with the passage of time. For example, attitudes about spending during the late 1980s 
differed dramatically one decade later within the same population, due to the sustained prosperity 
within the final four years of the millennium. Those same attitudes today differ vastly from just a few 
years earlier due to our sustained recession. Finally, data reflect their truthfulness by closeness to the 
phenomena. Secondary data have had at least one level of interpretation inserted between the event 
and its recording. Primary data are sought for their proximity to the truth and control over error. These 
cautions remind us to use care in designing data collection procedures and generalizing from results. 

Data are edited to ensure consistency across respondents and to locate omissions. In the case of 
survey methods, editing reduces errors in the recording, improves legibility, and clarifies unclear and 
inappropriate responses. Edited data are then put into a form that makes analysis possible. Because it 
is impractical to place raw data into a report, alphanumeric codes are used to reduce the responses to 
a more manageable system for storage and future processing. The codes follow various decision rules 
that the researcher has devised to assist with sorting, tabulating, and analyzing. Personal computers 
have made it possible to merge editing, coding, and data entry into fewer steps even when the final 
analysis may be run on a larger system. 


> Stage 5: Data Analysis and Interpretation 


Managers need information, not raw data. Researchers generate information by analyzing data after its 
collection. Data analysis usually involves reducing accumulated data to a manageable size, developing 
summaries, looking for patterns, and applying statistical techniques. Scaled responses on question- 
naires and experimental instruments often require the analyst to derive various functions, as well as to 
explore relationships among variables. Further, researchers must interpret these findings in light of the 
client’s research question or determine if the results are consistent with their hypotheses and theories. 
Increasingly, managers are asking research specialists to make recommendations based on their inter- 
pretation of the data. 

A modest example involves a market research firm that polls 2,000 people from its target population 
for a new generation of mobile phones. Each respondent will be asked four questions: 


1. “Do you prefer the convenience of Pocket-Phone over existing mobile phones?” 

2. “Are there transmission problems with Pocket-Phone?” 

3. “Is Pocket-Phone better suited to worldwide transmission than your existing mobile phone?” 

4. “Would cost alone persuade you to purchase Pocket-Phone?” 
The answers will produce 8,000 pieces of raw data. Reducing the data to a workable size will yield 
eight statistics: the percentage of yes and no answers to each question. When a half-dozen demographic 
questions about the respondents are added, the total amount of data easily triples. If the researcher 


scaled the four key questions rather than eliciting yes-no responses, the analysis would likely require 
more powerful statistical analysis than summarization. 


> Stage 6: Reporting the Results 


Finally, it is necessary to prepare a report and transmit the findings and recommendations to the man- 
ager for the intended purpose of decision making. The researcher adjusts the style and organization of 
the report according to the target audience, the occasion, and the purpose of the research. The results 
of applied research may be communicated via conference call, letter, written report, oral presentation, 
or some combination of any or all of these methods. Reports should be developed from the manager’s 
or information user’s perspective. The sophistication of the design and sampling plan or the software 
used to analyze the data may help to establish the researcher’s credibility, but in the end, the manager’s 
foremost concern is solving the management dilemma. Thus, the researcher must accurately assess 
the manager’s needs throughout the research process and incorporate this understanding into the final 
product, the research report. 
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The management decision maker occasionally shelves the research report without taking action. 
Inferior communication of results is a primary reason for this outcome. With this possibility in mind, a 


research specialist should strive for: 


¢ Insightful adaptation of the information to the client’s needs. 


¢ Careful choice of words in crafting interpretations, conclusions, and recommendations. 


Occasionally, organizational and environmental forces beyond the researcher’s control argue against the 

implementation of results. Such was the case in a study conducted for the Association of American Publish- 

ers, which needed an ad campaign to encourage people to read more books. The project, costing $125,000, 

found that only 13 percent of Americans buy general-interest books in stores. When the time came to commit 

$14 million to the campaign to raise book sales, the membership’s interest had faded and the project died.® 
At a minimum, a research report should contain the following: 


e An executive summary consisting of a synopsis of the problem, findings, and recommendations. 


¢ An overview of the research: the problem’s background, literature summary, methods and pro- 


cedures, and conclusions. 


¢ A section on implementation strategies for the recommendations. 


¢ A technical appendix with all the materials necessary to replicate the project. 


> Research Process Issues 


Although it is desirable for research to be thoroughly 
grounded in management decision priorities, studies can 
wander off target or be less effective than they should be. 


The Favored-Technique 
Syndrome 


Some researchers are method-bound. They recast the 
management question so that it is amenable to their fa- 
vorite methodology—a survey, for example. Others 
might prefer to emphasize the case study, while still oth- 
ers wouldn’t consider either approach. Not all research- 
ers are comfortable with experimental designs. The past 
reluctance of most social scientists to use experimental 
designs is believed to have retarded the development of 
scientific research in that arena. 

The availability of technique is an important factor 
in determining how research will be done or whether a 
given study can be done. Persons knowledgeable about 
and skilled in some techniques but not in others are too 
often blinded by their special competencies. Their con- 
cern for technique dominates the decisions concerning 
what will be studied (both investigative and measurement 
questions) and how (research design). 

Since the advent of total quality management 
(TQM), numerous, standardized customer satisfaction 
questionnaires have been developed. Jason may have 
done studies using these instruments for any number of 
his clients. Myra should be cautious. She must not let 
Jason steamroll her into the use of an instrument he has 


ot 


15% LOVE DYLAN 


8% LOVE HOW IT LOOKS 
IN THEIR APARTMENT 


4% LOVE LEARNING 
NEW THINGS 


LOVE THAT 
GIRLS LOVE IT? 


Why do people buy guitars? Or anything, really? We'r 
consumer behavior a: are. Sow 


Then we look for innovativ 


can get to the most meanin; 


Decipher is one research company that avoids fitting the client’s 
research problem to a specific technique. As the ad describes, 
Decipher first attempts to understand its client’s business. Then it 
looks for innovative ways to provide its client with relevant data for 
more meaningful insights. www.decipher.com 
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developed for another client, even though he might be very persuasive about its success in the past. 
Such a technique might not be appropriate for MindWriter’s search to resolve postpurchase service 
dissatisfaction. 


Company Database Strip-Mining 


The existence of a pool of information or a database can distract a manager, seemingly reducing the 
need for other research. As evidence of the research-as-expense-not-investment mentality mentioned in 
Chapter 1, managers frequently hear from superiors, “We should use the information we already have 
before collecting more.” Modern management information systems are capable of providing massive 
volumes of data. This is not the same as saying modern management information systems provide 
substantial knowledge. 

Each field in a database was originally created for a specific reason, a reason that may or may not 
be compatible with the management question facing the organization. The MindWriter service depart- 
ment’s database, for example, probably contains several fields about the type of problem, the location 
of the problem, the remedy used to correct the problem, and so forth. Jason and Sara can accumulate 
facts concerning the service, and they can match each service problem with a particular MindWriter 
model and production sequence (from a production database), and, using yet another database (gener- 
ated from warranty registration), they can match each problem to a name and address of an owner. But, 
having done all that, they still aren’t likely to know how a particular owner uses his or her laptop or how 
satisfied an owner was with MindWriter’s postpurchase service policies and practices. 

Mining management information databases is fashionable, and all types of organizations increas- 
ingly value the ability to extract meaningful information. While such data mining is often a starting 
point in decision-based research, rarely will such activity answer all management questions related to 
a particular management dilemma. 


Unresearchable Questions 


Not all management questions are researchable, and not all research questions are answerable. To be 
researchable, a question must be one for which observation or other data collection can provide the 
answer. Many questions cannot be answered on the basis of information alone. 

Questions of value and policy often must be weighed in management decisions. In the MetalWorks 
study, management may be asking, “Should we hold out for a liberalization of the seniority rules in our 
new labor negotiations?” While information can be brought to bear on this question, such additional 
considerations as “fairness to the workers” or “management’s right to manage” may be important to the 
decision. It may be possible for many of these questions of value to be transformed into questions of 
fact. Concerning “fairness to the workers,” one might first gather information from which to estimate 
the extent and degree to which workers will be affected by a rule change; then one could gather opin- 
ion statements by the workers about the fairness of seniority rules. Even so, substantial value elements 
remain. Questions left unanswered include “Should we argue for a policy that will adversely affect the 
security and well-being of older workers who are least equipped to cope with this adversity?” Even if 
a question can be answered by facts alone, it might not be researchable because currently accepted and 
tested procedures or techniques are inadequate. 


lll-Defined Management Problems 


Some categories of problems are so complex, value-laden, and bound by constraints that they prove to 
be intractable to traditional forms of analysis. These questions have characteristics that are virtually the 
opposite of those of well-defined problems. One author describes the differences like this: 


To the extent that a problem situation evokes a high level of agreement over a specified community of problem solvers 
regarding the referents of the attributes in which it is given, the operations that are permitted, and the consequences of 
those operations, it may be termed unambiguous or well defined with respect to that community. On the other hand, 
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to the extent that a problem evokes a highly variable set of responses concerning referents of attributes, permissible 
operations, and their consequences, it may be considered ill-defined or ambiguous with respect to that community.” 


Another author points out that ill-defined research questions are least susceptible to attack from 
quantitative research methods because such problems have too many interrelated facets for measure- 
ment to handle with accuracy.® Yet another authority suggests there are some research questions of 
this type for which methods do not presently exist or, if the methods were to be invented, still might 
not provide the data necessary to solve them.’ Novice researchers should avoid ill-defined problems. 
Even seasoned researchers will want to conduct a thorough exploratory study before proceeding with 
the latest approaches. 


Politically Motivated Research 


It is important to remember that a manager’s motivations for seeking research are not always 
obvious. Managers might express a genuine need for specific information on which to base a deci- 
sion. This is the ideal scenario for quality research. Sometimes, however, a research study may not 
really be desirable but is authorized anyway, chiefly because its presence may win approval for a 
certain manager’s pet idea. At other times, research may be authorized as a measure of personal 
protection for a decision maker in case he or she is criticized later. In these less-than-ideal cases, 
the researcher may find it more difficult to win the manager’s support for an appropriate research 
design. 


I 


1 Research originates in the decision process. A manager Exploration of the problem is accomplished through 


needs specific information for setting objectives, defin- 
ing tasks, finding the best strategy by which to carry 
out the tasks, or judging how well the strategy is being 
implemented. 

A dilemma-centered emphasis—the problem’s origin, 
selection, statement, exploration, and refinement— 
dominates the sequence of the research process. A 
management dilemma can originate in any aspect of an or- 
ganization. A decision to do research can be inappropriately 
driven by the availability of coveted tools and databases. To 
be researchable, a problem must be subject to observation 
or other forms of empirical data collection. 


How one structures the research question sets the direc- 
tion for the project. A management problem or opportunity 
can be formulated as a hierarchical sequence of questions. 
At the most general level is the management dilemma. This 
is translated into a management question and then into 

a research question—the major objective of the study. In 
turn, the research question is further expanded into inves- 
tigative questions. These questions represent the various 
facets of the problem to be solved, and they influence 
research design, including design strategy, data collec- 
tion planning, and sampling. At the most specific level are 
measurement questions that are answered by respondents 
in a survey or answered about each subject in an observa- 
tional study. 


familiarization with the available literature, interviews with ex- 
perts, focus groups, or some combination. Revision of the 
management or research questions is a desirable outcome 
of exploration and enhances the researcher’s understanding 
of the options available for developing a successful design. 


Budgets and value assessments determine whether most 
projects receive necessary funding. Their thorough docu- 
mentation is an integral part of the research proposal. Pro- 
posals are required for many research projects and should, 
at a minimum, describe the research question and the spe- 
cific task the research will undertake. 


Decisions concerning the type of study, the means of data 
collection, measurement, and sampling plans must be 
made when planning the design. Most researchers under- 
take sampling studies because of an interest in estimating 
population values or testing a statistical hypothesis. Care- 
fully constructed delimitations are essential for specifying an 
appropriate probability sample. Nonprobability samples are 
also used. 

Pilot tests are conducted to detect weaknesses in 
the study’s design, data collection instruments, and pro- 
cedures. Once the researcher is satisfied that the plan is 
sound, data collection begins. Data are collected, edited, 
coded, and prepared for analysis. 

Data analysis involves reduction, summarization, pattern 
examination, and the statistical evaluation of hypotheses. 
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A written report describing the study’s findings is used to 
transmit the results and recommendations to the intended 
decision maker. By cycling the conclusions back into the 
original problem, a new research iteration may begin, and 
findings may be applied. 

5 Several research process problems can diminish the value 
of research. Included in these are using a technique that 
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decision rule 82 
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investigative questions 79 Pumary data 62 
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Terms in Review 
1 Some questions are answerable by research and oth- 
ers are not. Using some management problems of your 
choosing, distinguish between them. 


2 Discuss the problems of trading off exploration and pilot 
testing under tight budgetary constraints. What are the 
immediate and long-term effects? 


3 Acompany is experiencing a poor inventory management 
situation and receives alternative research proposals. 
Proposal 1 is to use an audit of last year’s transactions as 
a basis for recommendations. Proposal 2 is to study and 
recommend changes to the procedures and systems used 
by the materials department. Discuss issues of evaluation 
in terms of: 


a Ex post facto versus prior evaluation. 
b Evaluation using option analysis and decision theory. 


Making Research Decisions 

4 It costs thousands of dollars to generate an application 
(app) for the Apple iPad. Just offering an app is not a 
guarantee of success. The most successful apps, sold as 
downloads, have to offer true functional value. Apple takes 
weeks to review an app proposal; only with Apple’s ap- 
proval can an app be officially offered for the iPad. If you 
were Apple, what research would you want to see within 
the proposal to approve a new app for the iPad? 


management question 79 


management-research question 


measurement questions 79 


is inappropriate for the information needed just because 
it is familiar or the researcher has experience with it; at- 
tempting to substitute data mining for research; focusing 
on an unresearchable question; failing to correctly define 
the management problem; and conducting politically 
motivated rather than management dilemma-motivated 
research. 


research process 76 
research question(s) 79 
sample 84 

secondary data 86 


target population 84 


5 Based on an analysis of the last six months’ sales, your 
boss notices that sales of beef products are declining in 
your chain’s restaurants. As beef entrée sales decline, so 
do profits. Fearing beef sales have declined due to several 
newspaper stories reporting E. coli contamination discov- 
ered at area grocery stores, he suggests a survey of area 
restaurants to see if the situation is pervasive. 


a What do you think of this research suggestion? 


b How, if at all, could you improve on your boss’s formula- 
tion of the research question? 


Bringing Research to Life 
6 What are the benefits to MindWriter if Henry and Associ- 
ates implements a pilot study? 


7 How can MindWriter’s existing database of service com- 
plaints be used to accumulate service problem information 
in advance of the proposed research. What information 
should be sought? 


From Concept to Practice 
8 Using Exhibit 4-1 and case examples from some research 
company’s website, discover how favored technique ap- 
proaches to research design dominate many firms’ activities. 


9 Using Exhibit 4-1, find a case study of a research example 
in which a clear statement of the management dilemma 
leads to a precise and actionable research. (Hint: Visit 


>chapter 4 The Research Process: An Overview 91 


research company websites —see Appendix 1a for com- that take nicotinamide riboside in comparatively high 
pany ideas—or use a search engine to find examples.) doses along with their high-fat meals burn more fat and 
From the Headlines are protected from obesity. They also become better 
10 Inarecent scientific discovery, a natural ingredient found runners because of muscles that now benefit from greater 
in milk can protect against obesity even while mice keep endurance. If you worked for the National Milk Producers 
on enjoying diets that are high in fat. The researchers, Federation, how would you create a research proposal to 
who reported their results in the Cell Metabolism, likened support a national ad campaign to promote the qualities of 
this finding to a new kind of vitamin. It appears that mice milk in a balanced diet, using this finding? 
FS 
ee Akron Children’s Hospital Mastering Teacher Leadership 
Calling Up Attendance NCRCC: Teeing Up a New Strategic Direction 


Covering Kids with Health Care & Ohio Lottery: Innovative Research Design Drives 
Winning 
Ramada Demonstrates Its Personal Best™ 


Goodyear’s Aquatred 


ee Donatos: Finding the New Pizza 


State Farm: Dangerous Intersections 
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Inquiring Minds Want to Know—NOW! 
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“You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 


chapter 5 


learningobjectives 
After reading this chapter, you should understand... 


The purposes and process of exploratory research. 

Two types and three levels of management decision-related secondary sources. 

Five types of external information and the five critical factors for evaluating the value of a source and its content. 
The process of using exploratory research to understand the management dilemma and work through the stages 
of analysis necessary to formulate the research question (and, ultimately, investigative questions and measurement 


questions). 


What is involved in internal data mining and how internal data-mining techniques differ from literature searches. 


6 GIt is critical to use serious business judgment 
about what types of information could possibly 
be useful and actionable for an organization. 
We have seen enormous resources expended 


on “data projects” that have no realistic chance 
of payoff. Indiscriminately boiling a data ocean 
seldom produces a breakthrough nugget. 9 9 


Blaise Heltai, general partner, 
NewVantage Partners 


bringingresearchtolife...<? 


Jason Henry presses the intercom button. “Sara, have 
you had a chance to summarize the transcripts from the 
CompleteCare call center?” 

“Those, as well as the summary of the complaint 
letters,” responds Sara as she strolls through the door 
to Jason’s office. 

Jason jerks at the closeness of her voice. “It’s 
unnerving how you anticipate what I want before I ask 
OVP ill 

“Not so tough,” chuckles Sara, enjoying seeing the 
unflappable Jason look a little shaken. “You did tell 
me you wanted them first thing this morning. And it 
is 8:05.” 

“What about those articles on measuring customer 
satisfaction in technological products?” 

“Tm human, Jason, not a robotic search engine. 
That’s the next thing on my iPhone to-do list.” 

“Tt seems logical that there might be special issues 
related to computers or other technical products when 
it comes to measuring satisfaction.” 

“But we both know that logic often doesn’t have 
a thing to do with reality. Do you want me to brief 


you on what I’ve discovered from the complaint 
correspondence and the transcripts, or would you rather 
read the summary?” 

“No, just leave it. I’ve set up an 8:30 phone interview 
with Sam Turnbull, the manager of the CompleteCare 
repair program. I need a few more minutes to review 
the questions I drafted last night.” 

“Fei, Wavy zilll 
interesting,” shares Sara as she hands the transcript 


transcript was _ particularly 
across the desk. “You might want to ask about this 
particular case. I’m sure he’1l remember it. 

“Tl just go start my search on whether technical 
products have differing measurement issues for 
satisfaction than the other industries we’ ve studied.” 

“Um, hm...” Jason responds as he peruses the 
transcript. Then he glances up and stops Sara before 
she glides out the door. “While you are at it, see if 
there is an industrywide study on laptop satisfaction, 
something we might use as a benchmark. And anything 
you can find about the special problems associated with 
laptop construction, operation, use patterns, or repairs.” 

elanronuts: 
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> A Search Strategy for Exploration 


Exploration is particularly useful when researchers lack a clear idea of the problems they will meet 
during the study. Through exploration researchers develop concepts more clearly, establish priori- 
ties, develop operational definitions, and improve the final research design. Exploration may also 
save time and money. If the problem is not as important as first thought, more formal studies can 
be canceled. 

Exploration serves other purposes as well. The area of investigation may be so new or so vague 
that a researcher needs to do an exploration just to learn something about the dilemma facing the 
manager. Important variables may not be known or thoroughly defined. Hypotheses for the research 
may be needed. Also, the researcher may explore to be sure it is practical to do a formal study in 
the area. 

Despite its obvious value, researchers and managers alike give exploration less attention than it 
deserves. There are strong pressures for quick answers. Moreover, exploration is sometimes linked to 
old biases about qualitative research: subjectiveness, nonrepresentativeness, and nonsystematic design. 
More realistically, exploration may save both time and money and should not be slighted. 

The exploratory phase search strategy usually comprises one or more of the following: 


e Discovery and analysis of secondary sources. 
e Published studies (usually focused on the results of surveys or on case studies featuring one 
or a few incidents). 
e Document analysis. 
¢ Retrieval of information from organization’s database(s). 


° Interviews with those knowledgeable about the problem or its possible solutions (called 
expert interviews). 


e Interviews with individuals involved with the problem (called individual depth interviews, 
or IDIs). 


e Group discussions with individuals involved with the problem or its possible solutions (includ- 
ing informal groups, as well as formal techniques such as focus groups or brainstorming). 


As the exploration process modeled with the management-research question hierarchy suggests 
(see Exhibit 5-1), exploration of secondary sources may be useful at any stage of the hierarchy. But 
most researchers find a review of secondary sources critical to moving from management question 
to research question. In moving from management question to research question, the researcher uses 
both internal and external secondary sources. Although most researchers would explore their internal 
archives first, here we will address external sources first. 

In this exploratory research phase of your project, your objective is to accomplish the 
following: 


e Expand your understanding of the management dilemma by looking for ways others have 
addressed and/or solved problems similar to your management dilemma or management 
question. 


e Gather background information on your topic to refine the research question. 
¢ Identify information that should be gathered to formulate investigative questions. 
e Identify sources for and actual questions that might be used as measurement questions. 


e Identify sources for and actual sample frames (lists of potential participants) that might be used 
in sample design. 


In most cases the exploration phase will begin with a literature search—a review of books as well 
as articles in journals or professional literature that relate to your management dilemma. This review 
should include high-quality Web-published materials. A literature search requires the use of the li- 
brary’s online catalog and one or more bibliographic databases or indexes. For some topics, it may be 
useful to consult a handbook or specialized encyclopedia first to establish a list of key terms, people, or 
events that have influenced your topic and also to determine what the major publications are and who 
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>Exhibit 5-1 Integration of Secondary Data into the Research Process 


Internal data or 
document search for 
solution ideas 


Internal data or 
document search for 
information topics 


Internal data or 
document search to 
understand the 
management dilemma 


Exploration 


Internal data or 
document search for 
possible sample 
frames 


Internal document 
search for sampling 
and research designs 


Research Design 


the foremost authors are. Other reference materials will be incorporated into your search strategy as 
needed. In general, this literature search has five steps: 


1. Define your management dilemma or management question. 

2. Consult encyclopedias, dictionaries, handbooks, and textbooks to identify key terms, people, or 
events relevant to your management dilemma or management question. 

3. Apply these key terms, names of people, or events in searching indexes, bibliographies, and the 
Web to identify specific secondary sources. 


4. Locate and review specific secondary sources for relevance to your management dilemma. 


5. Evaluate the value of each source and its content. 


The result of your literature search may be a solution to the management dilemma. In such a case, 
no further research is necessary. Often, however, the management question remains unresolved, so the 
decision to proceed generates a research proposal (see Appendix A). The resulting proposal covers at 
minimum a statement of the research question and a brief description of the proposed research meth- 
odology. The proposal summarizes the findings of the exploratory phase of the research, usually with 
a bibliography of secondary sources that have led to the decision to propose a formal research study. 
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In this chapter we will concentrate on the exploration phase of the project and focus on finding, 
selecting, and evaluating information in both printed and electronic formats. In some instances, re- 
searchers will discover the answer to their management dilemma in the results of a secondary search. 
A great exploration of secondary sources pays dividends—big ones—if a costly research project is 
deemed unnecessary. 


Levels of Information 


As you explore your problem or topic, you may consider many different types of information sources, 
some much more valuable than others. Information sources are generally categorized into three levels: 
(1) primary sources, (2) secondary sources, and (3) tertiary sources. 

Primary sources are original works of research or raw data without interpretation or pro- 
nouncements that represent an official opinion or position. Included among the primary sources are 
memos; letters; complete interviews or speeches (in audio, video, or written transcript formats); 
laws; regulations; court decisions or standards; and most government data, including census, eco- 
nomic, and labor data. Primary sources are always the most authoritative because the information 
has not been filtered or interpreted by a second party. Other internal sources of primary data are in- 
ventory records, personnel records, purchasing requisition forms, statistical process control charts, 
and similar data. 

Secondary sources are interpretations of primary data. Encyclopedias, textbooks, handbooks, 
magazine and newspaper articles, and most newscasts are considered secondary information sources. 
Indeed, nearly all reference materials fall into this category. Internally, sales analysis summaries and 
investor annual reports would be examples of secondary sources, because they are compiled from a 
variety of primary sources. To an outsider, however, the annual report is viewed as a primary source, 
because it represents the official position of the corporation. A firm searching for secondary sources can 
search either internally or externally, as Exhibit 5-2 depicts. 


>Exhibit 5-2 Secondary Sources for Developing the Question Hierarchy 


Exploration 


Internal 
Sources 
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Blogs: Gold Mine or Land Mine? 


Internet designers conceived a worldwide-accessible collection 
of documents, rather like a gargantuan library. What they could 
not foresee was the suppressed desire of millions to play the role 
of reporter, columnist, anchor, or analyst. Many blogs, defined 
as frequent, chronological publication of personal thoughts and 
Web links, started as personal online journals. Newer ones, how- 
ever, show more influential agendas. While the Internet has influ- 
enced research in numerous visible ways—how we collect data, 
process data, even report data—one unexpected influence of 
the Internet is this new and growing source of data. And it isn’t a 
source we can take lightly. 

While they vary widely, and are difficult to find, some esti- 
mates place the number of blogs at 1 billion, with the average 
blogger having 847 followers (more if they are professional). 
Technorati's latest State of the Blogosphere Report, drawn from 
a survey of more than 4,100 bloggers, indicates that most blog- 
gers (61 percent) are hobbyists, and while women are increas- 
ingly joining the ranks, most (69 percent) are men. Bloggers are 
an affluent group with 79 percent having college degrees, and 
most (80 percent) have been blogging for two or more years. 
Blogs are a favored source over traditional media for inspiration, 
product information and opinion, but traditional media are still 
preferred for news. And today bloggers are far more likely to get 
inspiration from each other, a 30 percent jump to 68 percent 


in just the last two years. While many believe that English will 
dominate, others believe Mandarin and German will always have 
a place in the blogosphere. This may change in the future, as 
there are already signs that young people are drifting to sites 
like Twitter. Many bloggers—pro (75 percent) and hobbyist 
(50 percent)—have Facebook accounts for their blogs. 

With the aid of search engines and RSS aggregators, a savvy 
company can monitor, respond to, and critique the ideas of 
hese vocal and influential individuals. Companies worried about 
disgruntled or careless employees revealing emerging product 
ideas or financial or legal strategies to competitors are estab- 
ishing strict blogging policies. Forward-thinking companies are 
aunching their own blogs, for both external and internal audi- 
ences, to encourage open, free-wheeling conversations about 
heir brands, products, employment policies, and problem- 
solving approaches. And researchers are challenged to spot 
emerging trends by mining this vast array of ideas and opinions 
traveling on the net at hyperspeed. 

Using the source evaluation process, how would you evalu- 
ate blog data? 

You can learn about many research-related blogs at http:/Awww 
.tomhcanderson.com/next-gen-market-research-top-blogs. 


http://technorati.com; www.pewinternet.org 


Tertiary sources may be interpretations of a secondary source but generally are represented by 
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indexes, bibliographies, and other finding aids (e.g., Internet search engines). 

From the beginning, it is important to remember that all information is not of equal value. As the 
source levels indicate, primary sources have more value than secondary sources, and secondary sources 
have more value than tertiary sources. In the opening vignette, Sara has read the MindWriter Complete 
Care call center transcripts (a primary source), and Jason suggests that she also check for articles 
related to laptop manufacture, repair, and satisfaction (all secondary sources). Sara’s summary of the 
transcript and letters is a secondary source. Henry & Associates’ program for MindWriter will hinge 
on Jason’s understanding of the current laptop repair scenario. If the information is essential to solving 
the management dilemma, it is wise to verify it in a primary source. That’s why Sara wants Jason to 
ask CompleteCare’s manager (primary source) for his take on the facts related to the disturbing call 
transcript she found. 


Types of Information Sources 


There are dozens of types of information sources, each with a special function. In this section we describe 
five of the information types used most by researchers at this phase of the project. 
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The U.S. government is the world’s 
largest source of data and information 
used by managers in all types of 
organizations. Learning how to navigate 
this government portal is a critical skill. 
Www.usa.gov 


Home | FAQs | Site Index | E-mail Us | Chat | FREE Publications 


Get E-mail Updates | Change Text Size [ Espaiiol | 


Search the Government ... SEARCH 1-800-FED-INFO (333-4636) 
USA.g p aa 
Government Made Easy 
Get Services News Explore Topics Find Government Agencies Contact Government 
Is Homeownership Right for You? POPULAR TOPICS 


Government Jobs 
Check out this FREE booklet to help 

you determine if you're ready to buy a 

home; calculate how much you can 

afford; and learn about the home 

buying process. 


& 


A-Z List of Agencies 


- 


Benefits, Grants, and 
Financial Aid 


# 


Government Auctions 


5. Unclaimed Money 
OG i2zas 
6 Just for You ... ® Featured App: QuitSTART (?) Poison Prevention 
* Citizens Get tips to help cravings, manage mood Do you know how to prevent or react to a 
. and stay smokefree. poisoning? 
* Kids More on QuitSTART app > Learn more > 
* Business 
* Visitors to the United States 
* Seniors = ? 
* Federal Employees ry 
pone SLES FIND MORE APPS CHECK MORE FAQS 


Connect with Government [i Facebook (5 Twitter i] RSS & Mobile i YouTube ie] Our Blog Ed share 


Home | About Us | Contact Us | Website Policies | Privacy | Link to Us 


USA.gov is the U.S. government's official web portal. 


Indexes and Bibliographies 


Indexes and bibliographies are the mainstay of any library because they help you identify and locate a 
single book or journal article from among the millions published. The single most important bibliogra- 
phy in any library is its online catalog. As with all other information types, there are many specialized 
indexes and bibliographies unique to business topics. These can be very useful in a literature search to 
find authors and titles of prior works on the topic of interest. 

Skill in searching bibliographic databases is essential for the business researcher. For the novice or 
less skilled, we provide two appendices at the end of this chapter. The first, “Bibliographic Database 
Searches,” reviews the process of searching. The second, “Advanced Searches,” reveals the more ad- 
vanced techniques of skilled searchers. 


Dictionaries 


Dictionaries are so ubiquitous that they probably need no explanation. We all use them to verify spelling 
or grammar usage or to define terms. In business, as in every field, there are many specialized dictionaries 
that define words, terms, or jargon unique to a discipline. Most of these specialized dictionaries include in 
their word lists information on people, events, or organizations that shape the discipline. They are also an 
excellent place to find acronyms. A growing number of dictionaries and glossaries (terms in a specialized 
field, area, or topic plus their definitions) are now available on the Web. Information from dictionaries 
and glossaries may be used to identify key terms for a search of an online or printed database. 


Encyclopedias 


Researchers use an encyclopedia to find background or historical information on a topic or to find 
names or terms that can enhance search results in other sources. For example, you might use an encyclo- 
pedia to find the date that Microsoft introduced Windows and then use that date to draw more informa- 
tion from an index to the time period. Encyclopedias are also helpful in identifying the experts in a field 


>chapter 5 Clarifying the Research Question through Secondary Data and Exploration 


Crossing the Deception Line? 


Most business intelligence activities are designed to help firms 
deal with the risks of operating in challenging environments. No- 
where is the issue of deception more in the news forefront than 
in the arena of business intelligence, an arena filled with actions 
designed to unearth the practices and processes of suppliers, 
competitors, employees, and even directors. 

One recent example in the news involves the actions of an in- 
telligence firm, Diligence, Inc. Diligence was hired by well-known 
Washington, D.C., lobbying firm Barbour Griffith & Rogers (BGR) 
to discover competitive information for one of its clients, a Russian 
conglomerate, about that firm’s archrival IPOC International Growth 
Fund Ltd. (IPOC). The task as described here is not inherently de- 
ceitful. Worldwide, employees are charged with such tasks on a 
daily basis. You could approach such a task in numerous ways: 
conduct a comprehensive literature search, talk with industry ex- 
perts, interview former employees, monitor competitor publications, 
or attend presentations by their executives, and so forth. Diligence 
chose a covert use of deception to achieve its task. 

Diligence first discovered that IPOC was the client of a repu- 
table accounting firm KPMG. Diligence researched KPMG’s 


employees, seeking an individual who might leak audit informa- 
tion. The search revealed the identity of a British-born accoun- 
tant. Diligence sent an employee, masquerading as a British 
intelligence officer needing assistance on a matter of top British 
national security, to recruit the assistance of the accountant. After 
several weeks of luncheon meetings, the Diligence employee 
started inquiring about IPOC’s audit. Ultimately, the KPMG em- 
ployee shared proprietary information with the supposed under- 
cover British security agent. KPMG sued Diligence and the case 
was settled out of court with Diligence paying KPMG $1.7 million. 
IPOC’s lawsuit against Diligence and its client is still pending. 

Most of us recognize the demarcation between deception 
and fraud. But what if in a casual social conversation with an 
acquaintance over lunch you inadvertently reveal what some firm 
wants to know? Some researchers say that deception is never 
appropriate. Other researchers claim that deception is appro- 
priate to mask the sponsor’s identity and the purpose of the 
research. Where do you stand on the issue of deception in busi- 
ness intelligence gathering? 


www.businessweek.com 


and the key writings on any topic. One example of an encyclopedia is the Online TDM Encyclopedia 
published by the Victoria Transportation Policy Institute. 


The Online TDM Encyclopedia is a comprehensive source of information about innovative management solutions to 
transportation problems. The Encyclopedia provides detailed information on dozens of Transportation Demand Man- 
agement (TDM) strategies, plus chapters on their planning, evaluation and implementation. It can help you view trans- 
portation problems from a new perspective, and expand the range of possible solutions to apply.! 
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Another example drawn from the area of finance is the Encyclopedia of Private Equity and Venture 
Capital, published by VC Experts, Inc., a provider of expertise and opportunity in private equity and 
venture capital.’ 


Handbooks 


A handbook is a collection of facts unique to a topic. Handbooks often include statistics, directory in- 
formation, a glossary of terms, and other data such as laws and regulations essential to a field. The best 
handbooks include source references for the facts they present. The Statistical Abstract of the United 
States is probably the most valuable and frequently used handbook available. It contains an extensive 
variety of facts, an excellent and detailed index, and a gateway to even more in-depth data for every table 
included. One handbook with which students and managers alike are familiar is the Occupational Outlook 
Handbook published by the U.S. Bureau of Labor Statistics. In it you can find details about many business 
occupations.* Many handbooks are quite specialized, such as the one published by the Potato Association 
of America. It reveals not only consumption patterns but also potato growing and processing statistics.* 

One of the most important handbooks, especially in the business-to-business arena, is the North 
American Industry Classification System, United States (NAICS). Jointly designed with Canada and 
Mexico to provide comparability in business statistics throughout North America, especially as new 
businesses and new business sectors develop, this classification system of all businesses replaced the 
Standard Industrial Classification in 1997.5 
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Surfing the Deep Web 


According to Rider University librarian Robert Lackie, “Although 
many popular search engines boast about their ability to index in- 
formation on the Web, some of the Web's information is invisible to 
their searching spiders.” Why is it invisible? Searchwise’s president 
Chris Sherman explains, “The most basic reason is that there are 
no links pointing to a page that a search engine spider can follow. 
Or, a page may be made up of data types that search engines 
don’t index—graphics, CGI scripts, or Macromedia Flash, for ex- 
ample.” Today, non-HTML pages, dynamically created pages, and 
even scripted pages that were invisible a few years ago are now 
regularly found by search engines Google, Yahoo!, and more. What 
those search engines still cannot “see” is the content of specialized 
searchable databases on the Web (e.g., specialized government 
databases) and those the search engines choose to exclude be- 
cause they view them as of little use to the searching public. Bright- 
Planet, a compiler of more than 35,000 searchable datalbases—the 
largest on the Internet—estimates that the invisible Web may be as 
much as 500 times the size of the searchable Web. 

Lackie suggests we access this hidden content with just 
a few tools: directories and portals, searchable sites (Some of 
which likely are available through your university library), free 
Web databases, and a few general and many specialized search 
engines. Here is just a sample: 


ipl2 (http://www. ipl.org/)—This research portal maintained 
by librarians merged the collections of resources from the 
Internet Public Library (IPL) and the Librarians’ Internet Index 
(LIl) websites. 


Directories 


FreePint (www.freepint.com/)—This search tool offers access 
to a compilation of links to the search interfaces of resources. 
InfoMine (infomine.ucr.edu)—This directory offers a scholarly 
resource collection that includes tens of thousands of sites. 
About.com (http://www.about.com/)—This portal amasses 
its content by the passionate interests of 570 expert guides; 
more than 34 million people visit its neatly organized content 
of thousands of topics, including Invisible Web. 


CompletePlanet (http:/Awww.completeplanet.com/) — 
BrightPlanet’s site contains more than 70,000 searchable 
databases and specialty search engines. 

FindArticles (http://www. findarticles.com/)—This specialized 
database contains more than 10 million articles from “leading 
academic, industry and general interest publications.” 

Super Searchers Web Page (http://www. infotoday.conV/ 
supersearchers/)—This site indexes a growing collection of links 
to subject-specific Web resources in global business, primary 
research, mergers/acauisitions, news, investment, business, 
entrepreneurial research, and legal information resources. 


AOL Video (on.aol.com)—This audio/video search engine 
indexes multimedia formats, including Windows Media, 
Real, QuickTime, and MP8s. 


Never stop your information search until you’ve looked for 
specialized databases that might be invisible. Such sources 
could be hiding a treasure trove of information perfect for your 
particular research question. 


Directories are used for finding names and addresses as well as other data. Although many are avail- 
able and useful in printed format, directories in digitized format that can be searched by certain char- 
acteristics or sorted and then downloaded are far more useful. Many are available free through the 
Web, but the most comprehensive directories are proprietary (i.e., must be purchased). An especially 
useful directory available in most libraries in either print or electronic format is the Encyclopedia of 
Associations (called Associations Unlimited on the Web), which provides a list of public and pro- 
fessional organizations plus their locations and contact numbers.° New York AMA Communications 
Services, Inc., publishes the Green Book, a Guide for Buyers of Marketing Research Services." 


Evaluating Information Sources 


A researcher using secondary sources, especially if drawn from the Internet, will want to conduct a 
source evaluation. Researchers should evaluate and select information sources based on five factors 
that can be applied to any type of source, whether printed or electronic. These are: 


e Purpose—the explicit or hidden agenda of the information source. 


° Scope—the breadth and depth of topic coverage, including time period, geographic limitations, 
and the criteria for information inclusion. 
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° Authority—the level of the data (primary, secondary, tertiary) and the credentials of the source 
author(s). 

e Audience—the characteristics and background of the people or groups for whom the source was 
created. 

e Format—how the information is presented and the degree of ease of locating specific 
information within the source. 


Sara is about to embark on an Internet search for various types of information. Exhibit 5-3 summarizes 
the critical questions she should ask when applying these source evaluation factors to the evaluation of 
Internet sources she discovers. 


>Exhibit 5-3 Evaluating Websites as Information Sources 


Evaluation Factor Questions to Answer 


Purpose ¢ Why does the site exist? 


¢ How evident is the purpose it is trying to convey? 
¢ Does it achieve its purpose? 


How does its purpose affect the type and bias of information presented? 


Scope ¢ How old is the information? 


¢ How often is it updated? 

¢« How much information is available? 

¢ ls it selective or comprehensive? 

¢ What are the criteria for inclusion? 

¢ If applicable, what geographic area or time period or language does it cover? 
¢ How does the information presented compare with that on similar sites? 

¢ ls ita series of links only (a metasite), or is there added value? 

« What is the nature of the added value? 


¢ What information did you expect to find that was missing? 


¢ ls the site self-contained, or does it link to other websites? 


Format ¢ How quickly can you find needed information? 
¢ How easy is the site to use? Is it intuitive? 

¢ Does it load quickly? 

¢ Is the design appealing? 

¢ Are there navigation buttons? 

¢ ls there a site map or search button? 

¢ Is there an easily identifiable Help button? 

¢ Is Help helpful? 

¢ Are pages in ASCII or graphic format? 


¢ Is the information downloadable into a spreadsheet or word processing program, if desired? 
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The purpose of early exploration is to help the researcher understand the management dilemma and 
develop the management question. Later stages of exploration are designed to develop the research 
question and ultimately the investigative and measurement questions. 


> Mining Internal Sources 


The term data mining describes the process of discovering knowledge from databases stored in 
data marts or data warehouses. The purpose of data mining is to identify valid, novel, useful, and 
ultimately understandable patterns in data.* Similar to traditional mining, where we search beneath 
the surface for valuable ore, data mining searches large databases for indispensable information 
for managing an organization. Both require sifting a large amount of material to discover a profit- 
able vein. Data mining is a useful tool, an approach that combines exploration and discovery with 
confirmatory analysis. 

An organization’s own internal historical data are often an underutilized source of information in the 
exploratory phase. Due to employee turnover, the researcher may lack knowledge that such historical 
data exist; or, based on time or budget constraints and the lack of an organized archive, the researcher 
may choose to ignore such data. Although digging through data archives can be as simplistic as sorting 
through a file containing past patient records or inventory shipping manifests, or rereading company 
reports and management-authored memos that have grown dusty with age, we will concentrate the 
remainder of our discussion on more sophisticated structures and techniques. 

A data warehouse is an electronic repository for databases that organizes large volumes of 
data into categories to facilitate retrieval, interpretation, and sorting by end users. The data ware- 
house provides an accessible archive to support dynamic organizational intelligence applications. 
The key words here are dynamically accessible. Data warehouses that offer archaic methods for 
data retrieval are seldom used. Data in a data warehouse must 
be continually updated to ensure that managers have access 
to data appropriate for real-time decisions. In a data ware- 
house, the contents of departmental computers are duplicated 
in a central repository where standard architecture and con- 
sistent data definitions are applied. These data are available 
to departments or cross-functional teams for direct analysis 
or through intermediate storage facilities or data marts that 
compile locally required information. The entire system must 
be constructed for integration and compatibility among the 
different data marts. 

The more accessible the databases that comprise the data 
warehouse, the more likely a researcher will use such databases 
to reveal patterns. Thus, researchers are more likely to mine 
electronic databases than paper ones. It will be useful to re- 
member that data in a data warehouse were once primary data, 
collected for a specific purpose. When researchers data-mine a 
company’s data warehouse, all the data contained within that 
database have become secondary data. The patterns revealed 
will be used for purposes other than those originally intended. 
For example, in an archive of sales invoices, we have a wealth 
of data about what was sold, how much of each item or ser- 
vice, at what price level, to whom, and where and when and 
how the products were shipped. Initially the company gener- 
ated the sales invoice to facilitate the process of getting paid for 
Numerous companies build large consumer purchase behavior — the items shipped. When a researcher mines that sales invoice 
databases by collecting transaction data made via store- archive, the search is for patterns of sales, by product, category, 
owned credit programs or frequent purchase loyalty idenitifica- : ; i 
tion programs not linked directly with payment plans. Studying region of the country or world, p eats level, shipping methods, 
such data can reveal the likely success of anew product intro- and so forth. Therefore, data mining forms a bridge between 
duction or the sales lift effect of a price incentive. primary and secondary data. 
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How Will Cloud Computing Affect Research? 


Cloud computing refers to “a computing environment where 
data and services reside in scalable data centers accessible over 
the Internet.” Most of us are unaware of this cloud, but we may 
be using it if we subscribe to a Web-based e-mail service such 
as G-mail. Of course, the influence of cloud computing extends 
much farther. While cloud computing is different than SaaS (soft- 
ware as a service, where organizations access applications via 
the Internet but their data, documents, etc., reside on their own 
computers), many expanding clouds include both the infrastruc- 
ture for data warehousing along with unique and customizable 
applications. Amazon, Dell, Google, IBM, Microsoft, HP, Yahoo, 
and Salesforce.com are some of the bigger players, but the 
cloud is dense with smaller players, too. 

Amazon's Elastic Compute Cloud, known as Amazon EC2, 
claims to change “the economics of computing by allowing [the 
organization] to pay only for [server] capacity that [it] actually 
uses.” What that means for research is that data mined from 
all research sources (customer transactions, employee evalua- 
tions, financial records, surveys, etc.) may no longer be resident 
on an individual organization’s servers, but rather resident on 
the Web. We can already see this occuring with sites such as 
SurveyMonkey.com. A company can design and field multiple 
online surveys, and the data from each is collected and resident 
with SurveyMonkey (or potentially outsourced to another cloud 
provider). SurveyMonkey allows you to analyse the data online 


as the survey progresses—in real time—and share access to 
that data or its analysis with anyone. 

The big advantage of cloud computing is that it is “paid 
for only when it is activated and can scale as large or small as 
needed at the time it is needed.” As Salesforce.com claims for 
its cloud, Force.com, “There’s no hardware to purchase, scale, 
and maintain, no operating systems, database servers, or ap- 
plication servers to install, no consultants and staff to manage it 
all, and no need for upgrades.” Its second strength is collabora- 
tion through widely dispersed access that many feel stimulates 
innovation. For example, with Microsoft’s cloud, called SkyDrive, 
claims “you'll never be without the documents, notes, photos, 
and videos that matter to you. Store anything on your SkyDrive, 
and it’s automatically available from your trusted devices—no 
syncing or cables needed.” And you can share files with others 
whether they are on their phone, Mac, or PC. 

While once the cloud’s greatest perceived weakness was 
security and lack of control of proprietary information, Adam 
Selipesky, Amazon’s vice president of product marketing, sales, 
and product management claimed at the Amazon Web Service 
Summit 2012 that “People are getting a glimpse of a future in 
which most enterprises will not own or operate data centers, 
and those that do will have small, special-purpose data centers.” 


aws.amazon.com/ec2; www.skydrive.com; 
www.SurveyMonkey.com; www.salesforce.com/platform 


Traditional database queries are unidimensional and historical—for example, “How much beer 
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was sold during December in the Sacramento area?” In contrast, data mining attempts to discover 
patterns and trends in the data and to infer rules from these patterns. For example, an analysis of 
retail sales by Sacramento FastShop identified products that are often purchased together—like beer 
and diapers—although they may appear to be unrelated. With the rules discovered from the data 
mining, a manager is able to support, review, and/or examine alternative courses of action for solv- 
ing a management dilemma, alternatives that may later be studied further in the collection of new 
primary data. 


Evolution of Data Mining 


The complex algorithms used in data mining have existed for more than two decades. The U.S. govern- 
ment has employed customized data-mining software using neural networks, fuzzy logic, and pattern 
recognition to spot tax fraud, eavesdrop on foreign communications, and process satellite imagery.’ 
Until recently, these tools have been available only to very large corporations or agencies due to their 
high costs. However, this is rapidly changing. 

In the evolution from business data to information, each new step has built on previous ones. For 
example, large database storage is crucial to the success of data mining. The four stages listed in 
Exhibit 5-4 were revolutionary because each allowed new management questions to be answered 
accurately and quickly.!° 
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>Exhibit 5-4 The Evolution of Data Mining 


Evolutionary Step Investigative Question Enabling Technologies Characteristics 


Computers, tapes, disks 


Retrospective, static data 
delivery 


Data Collection (1960s) 


“What was my average total 
revenue over the last five 
years?” 


Online analytic processing 
(OLAP), multidimensional da- 
tabases, data warehouses 


Retrospective, dynamic data 
delivery at multiple levels 


“What were unit sales in 
California last December? 
Drill down to Sacramento.” 


Data Navigation (1990s) 


The process of extracting information from data has been done in some industries for years. Insur- 
ance companies often compete by finding small market segments where the premiums paid greatly out- 
weigh the risks. They then issue specially priced policies to a particular segment with profitable results. 
However, two problems have limited the effectiveness of this process: getting the data has been both 
difficult and expensive, and processing it into information has taken time—making it historical rather 
than predictive. Now, instead of incurring high data collection costs to resolve management questions, 
secondary data are available to assist the manager’s decision making. It was State Farm Insurance’s 
ability to mine its extensive nationwide database of accident locations and conditions at intersections 
that allowed it to identify high-risk intersections and then plan a primary data study to determine alter- 
natives to modify such intersections. 

Functional areas of management and select industries are currently driving data-mining projects: 
marketing, customer service, administrative/financial analysis, sales, manual distribution, insurance, 
fraud detection, and network management.'! Data-mining technology provides two unique capabilities 
to the researcher or manager: pattern discovery and prediction. 


Pattern Discovery 


Data-mining tools can be programmed to sweep regularly through databases and identify previously 
hidden patterns. An example of pattern discovery is the detection of stolen credit cards based on analy- 
sis of credit card transaction records. MasterCard processes 12 million transactions daily and uses 
data mining to detect fraud.'* Other uses include finding retail purchase patterns (used for inventory 
management), identifying call center volume fluctuations (used for staffing), and locating anomalous 
data that could represent data entry errors (used to evaluate the need for training, employee evaluation, 
or security). 


Predicting Trends and Behaviors 


A typical example of a predictive problem is targeted marketing. Using data from past promotional 
mailings to identify the targets most likely to maximize return on investment can make future mailings 
more effective. Bank of America and Mellon Bank both use data-mining software to pinpoint market- 
ing programs that attract high-margin, low-risk customers. Bank of America focuses on credit lines and 
retail lending; Mellon Bank has used data mining to optimize its home equity line of credit marketing 
to existing customers.'? Other predictive problems include forecasting bankruptcy and loan default and 
finding population segments with similar responses to a given stimulus. Data-mining tools also can be 
used to build risk models for a specific market, such as discovering the top 10 most significant buying 
trends each week. 
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Mining the Web for Feelings? 


Data mining, or pattern extraction from large amounts of data, is 
great for sifting through numbers, but can it interpret “feelings”? 

Alex Wright asked this question in a recent New York Times 
article about the explosion of blogs and social networks with the 
accompanying deluge of opinions. Bloggers can have a huge im- 
pact in influencing views on politics, business, entertainment, and 
sports. Since blogging and networking are about expressing and 
listening to opinion, tools for emotive text analysis are essential. 
The problem for analysts is to extract human emotion from reviews 
and recommendations and translate them into usable data. 

The emerging field of sentiment analysis and opinion mining 
provides us with a means of applying a computational treatment 
to opinion, sentiment, and subjectivity in textual form. Its goal is to 
decode a speaker or writer’s attitude on some topic. This might 
be an evaluation, the writer’s affective state, or the emotional ef- 
fect the writer creates in readers. A business might ask, for exam- 
ple, “Are our preliminary product reviews positive or negative?” 
Or, “Are blog posts running in favor of a particular policy position 
that affects environmental costs?” “The mistake people make is 
they just listen for brands and miss all the conversations,” says 
Frank Cotignola, consumer insights manager at Kraft Foods. 

How does it work? Software scans for keywords and cate- 
gorizes a statement as positive or negative, using a simple binary 
assignment (“success” is good, “failure” is bad). But, as Wright 
explains, the simplicity “fails to capture the subtleties that bring 
human language to life: irony, sarcasm, slang and other idiom- 
atic expressions.” The difficulty for most sentiment algorithms 
is the reliance on simple keywords to express complex feelings 
about a product, movie, or service. It is the cultural factors, lin- 
guistic shading, and contexts that make turning written text into 
digitized sentiment difficult. 

Speaking at the 2012 Sentiment Analysis Symposium, Cath- 
erine van Zuylen, vice president of products at Attensity, a social 


Data-Mining Process 


e Sample: Decide between census and sample data. 
e Explore: Identify relationships within the data. 
° Modify: Modify or transform data. 


analytics software vendor, identified seven difficult comment- 
analysis problems: False Negatives: “| cried so hard; it was one 
of the best movies ever!”; Relative sentiment: “| bought a Honda 
Civic not the Toyota Corrola.”; Compound sentiment: “| loved the 
way the car looked but hated the ride.”; Conditional sentiment: “| 
was about to cancel my membership, but then they offered me a 
coupon for a free rental.”; Scoring Sentiment: “| liked the movie.” 
versus “I really like the movie.” versus “I loved it!”; Interpreting 
Sentiment Modifiers (Emoticons): “| bought an iPhone today :-)” 
or “Gotta love the cable company;-<”; Interpreting International 
Sentiment: unique emoticons by country, effusive or grandiose 
language versus more reserved language due to culture. 

Bo Pang, one of the first academics and software develop- 
ers in the field explains that to understand the true intent of a 
statement, filters for intensity and subjectivity should be added 
to polarity (good-bad/positive-negative). Intensity refers to the 
level of emotion being expressed and subjectivity is the degree 
of source impartiality. Multi-way scales are also used to expand 
polarity scaling, sometimes with 3- to 5-star ratings. 

As businesses look to the field of sentiment analysis to auto- 
mate filtering, understanding, and identifying relevant emotional 
content, they venture beyond traditional fact-based services. 
One such subscription service responded to the demand to find 
trends in opinions about products, services, or news topics. It 
allows customers to monitor blogs, news articles, online forums, 
and social networking sites. Other services track business top- 
ics, newspaper editorials, movie releases, and Twitter. 

What broader ethical issues involving questions of privacy 
and vulnerability to manipulation do you see? How are your an- 
swers reconciled with the potential economic impact of senti- 
ment analysis? 


www.attensity.com 


Data mining, as depicted in Exhibit 5-5, involves a five-step process:'* 


¢ Model: Develop a model that explains the data relationships. 


e Assess: Test the model’s accuracy. 


To better visualize the connections between the techniques just described and the process steps 


listed in this section, you may want to download a demonstration version of data-mining software from 
the Internet. 
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>Exhibit 5-5 Data-Mining Process 


name YES/NO 
Explore 
Clustering, Variable Data 

Modify Factor, Selection, Transformation 

Correspondence Creation 
Model 
Assess Model 

Assessment 
Sample 


Exhibit 5-5 suggests that the researcher must decide whether to use the entire data set or a sample of 
the data.'° If the data set in question is not large, if the processing power is high, or if it is important to 
understand patterns for every record in the database, sampling should not be done. However, if the data 
warehouse is very large (terabytes of data), the processing power is limited, or speed is more important 
than complete analysis, it is wise to draw a sample. In some instances, researchers may use a data mart 
for their sample—with local data that are appropriate for their geography. Alternatively, the researcher 
may select an appropriate sampling technique. Since fast turnaround for decisions is often more impor- 
tant than absolute accuracy, sampling is appropriate. 

If general patterns exist in the data as a whole, these patterns will be found in the sample. If a niche 
is so tiny that it is not represented in a sample yet is so important that it influences the big picture, it will 
be found using exploratory data analysis (EDA), which we explore in Chapter 16. 


Explore 


After the data are sampled, the next step is to explore them visually or numerically for trends or groups. 
Both visual and statistical exploration (data visualization) can be used to identify trends. The researcher 
also looks for outliers to see if the data need to be cleaned, cases need to be dropped, or a larger sample 
needs to be drawn. 


Modify 


Based on the discoveries in the exploration phase, the data may require modification. Clustering, 
fractal-based transformation, and the application of fuzzy logic are completed during this phase as 
appropriate. A data reduction program, such as factor analysis, correspondence analysis, or clustering, 
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OdinText: Text Analytics for the Do-It-Yourself Researcher 


With the advent of large databases of unstructured data, re- 
searchers have been looking for a tool that would allow them to 
do more advanced text analytics on their own. Anderson Analyt- 
ics new software-as-a service (SaaS) product, OdinText, offers 
to deliver. 

IBM’s senior research manager, James Newswanger, de- 
fines text analytics as “the process of taking in [unstructured] 
content, using software to mine the content for meaning, and 
offering a presentation interface that allows researchers to find 
insight. Some researchers need to find basic themes in text, 
[while] some need to identify sentiment associated with themes.” 

Tom H. C. Anderson, founder and managing partner of An- 
derson Analytics and a thought leader in the area of text analytics, 
indicates “my company’s role [has] been evolving from a more 
full-service approach [to research and analysis] to helping clients 
take a hands-on approach to unstructured data analytics.” Un- 
structured data has been increasing with the explosion of blogs, 
social media sites, online surveys, use of email, and increased call- 
center activity. Text analytics is evolving from a “more linguistic 
approach—which has fallen out of favor a bit now—to statistical 
and machine learning methods, which seem to be proving more 
effective,” especially for large amounts of data. 

“OdinText,” shared Anderson, “helps clients monitor cus- 
tomer comments drawing their attention to new important 
trends. The software identifies these important issues and also 
calculates sentiment. Users find that the software is helpful in 


filtering out the noise so common in large amounts of text data. 
Most users upload their own data to a custom project folder 
on the OdinText server. The software then codes and hosts 
the data and acts as the front-end user interface allowing them 
to do various analyses, etc. The software was developed es- 
pecially for market research, but also provides powerful visual- 
ization capabilities to help users explore and understand their 
data quickly.” 

One user of OdinText is KODAK Gallery. KODAK Gallery 
turned to OdinText to analyze 1,500-2,000 weekly open-end 
responses from two ongoing customer satisfaction tracking 
studies, without sufficient time or resources for traditional cod- 
ing. “The goal was to find a way to categorize these verbatim 
responses into logical topic areas that could serve as a diag- 
nostic indicator for key metric trend changes and guide further 
investigation,” shared director of research at KODAK Gallery, 
Lori Tarabek. 

Anderson, who fears social media has been overhyped be- 
ieves social media monitoring is just one source of text data. 
“Most firms have a wealth of rich unstructured data within their 
organization already that they need to understand.” OdinText 
was developed to handle these large unstructured data sets that 
were often cumbersome to analyze with typical statistical pack- 
ages and impossible to code by hand. 


www.odintext.com 
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may be used. If important constructs are discovered, new factors may be introduced to categorize the 
data into these groups. In addition, variables based on combinations of existing variables may be added, 
recoded, transformed, or dropped. 

At times, descriptive segmentation of the data is all that is required to answer the investigative ques- 
tion. However, if a complex predictive model is needed, the researcher will move to the next step of 
the process. 


Model 


Once the data are prepared, construction of a model begins. Modeling techniques in data mining in- 
clude neural networks as well as decision tree, sequence-based, classification and estimation, and 
genetic-based models. 


Assess 


The final step in data mining is to assess the model to estimate how well it performs. A common 
method of assessment involves applying a portion of data that was not used during the sampling 
stage. If the model is valid, it will work for this “holdout” sample. Another way to test a model is 
to run the model against known data. For example, if you know which customers in a file have high 
loyalty and your model predicts loyalty, you can check to see whether the model has selected these 
customers accurately. 
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> The Question Hierarchy: How Ambiguous 
Questions Become Actionable Research 


The process we call the management-research question hierarchy is designed to move the researcher 
through various levels of questions, each with a specific function within the overall business research 
process. This multistep process is presented in Exhibit 5-6 and in the example in Exhibit 5-7. The role 
of exploration in this process is depicted in Exhibit 5-8. 


The Management Question 


Management questions are the restatement of the management dilemma in question form. The man- 
agement questions that evolve from the management dilemma are too numerous to list, but we can 
categorize them (see Exhibit 5-9). No matter how the management question is defined, many research 
directions can be taken. A specific question can lead to many studies. Therefore, it is the joint respon- 
sibility of the researcher and the marketer to choose the most productive project. 

Assume, for example, a business researcher is hired to help the new management of a bank. The 
president is concerned about erosion of the bank’s profitability (the management dilemma) and wants 
to turn this situation around. BankChoice is the oldest and largest of three banks in a city with a popu- 
lation of about 80,000. Profits have stagnated in recent years. The president and the researcher discuss 
the problem facing the organization and settle on this management question: “How can we improve 
our profit picture?” 

The management question does not specify what kind of business research is to be done. This ques- 
tion is strictly managerial in thrust. It implies that the bank’s management faces the task of developing 
a strategy for increasing profits. The question is broad. Notice that it doesn’t indicate whether man- 
agement should increase profits via increased deposits, downsizing of personnel, outsourcing of the 
payroll function, or some other means. 


>Exhibit 5-6 Management-Research Question Hierarchy 


Management What is the recommended course of 
Decision action, given the research findings? 


Measurement What should be asked or observed to obtain the 
Questions information the manager needs? 


Investigative 4 What does the manager need to know to choose the 


Questions best alternative from the available courses of action? 


What plausible courses of action are available to 
Research management to correct the problem or take advantage 
Question(s) of the opportunity, and which should be considered? 


Management How can management eliminate the negative symptoms? 
Question How can management fully capitalize on an opportunity? 


Management What symptoms cause management concern? What 
Dilemma environmental stimuli raise management interest? 
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>Exhibit 5-7 SalePro’s Management-Research Question Hierarchy 

Declining sales is one of the most common symptoms serving as a stimulus for a research project, especially a continuing pattern that is 
unexplained. SalePro, a large manufacturer of industrial goods, faces this situation. Exploration (1) reveals that sales, in fact, should not be 
declining in the South and Northeast. Environmental factors there are as favorable as in the growing regions. Subsequent exploration (2, 3) 
leads management to believe that the problem is in one of three areas: salesperson compensation, product formulation, or trade advertis- 
ing. Further exploration (4) has SalePro management narrowing the focus of its research to alternative ways to alter the sales compensation 
system, which (5) leads to a survey of all sales personnel in the affected regions. 


w¥ Why are our sales declining in the South and Northeast, while sales are booming in the Southwest? 


ws How can we improve sales in the South and Northeast? 


Should we introduce a 2 percent incentive commission-based compensation system on all sales 
over quota for salespeople in the South and Northeast or a 5-percent-of-profit regional bonus to 
the region that increases sales by 10 percent over quota (to be shared proportionately among the 
salespeople in the region)? Should we modify the product formula for distribution in the South 
and Northeast? Should we increase the level of advertising via trade publications in South and 
Northeast editions? 


implement the compensation change? What is the likelihood that current customer satisfaction 
in these regions will decrease? What is the likelihood that future sales to existing customers 
will be lost? 


ao, What is the likelihood that we will lose excellent salespeople in the South and Northeast if we 


change your compensation to a commission-based system compared to the current salary 
system. For each outcome, indicate a number between 1 and 7 where 7 = extreme concern, 
4 = neither concerned nor unconcerned, and 1 = no concern at all. 

Lack of predictability of monthly pay. 

Increased internal competition for sales prospects. 

Reduced time for postsale servicing of customer needs. 

Reduced incentive for postsale servicing of customer needs. 


ire Please rate your level of concern for each of the following outcomes if management were to 


Further discussion between the bank president and the business researcher shows there are really 
two questions to be answered. The problem of low deposit growth is linked to concerns of a competitive 
nature. While lowered deposits directly affect profits, another part of the profit weakness is associated 
with negative factors within the organization that are increasing customer complaints. The qualified 
researcher knows that the management question as originally stated is too broad to guide a definitive 
business research project. As a starting point, the broadly worded question is fine, but BankChoice will 
want to refine its management question into these more specific subquestions: 


¢ How can we improve deposits? 


¢ How can we improve internal operations that currently result in customer complaints? 


This separation of the management question into two subquestions may not have occurred without a 
discussion between the researcher and the manager. 


Exploration 


BankChoice has done no formal business research in the past. It has little specific information about 
competitors or customers and has not analyzed its internal operations. To move forward in the 
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>Exhibit 5-8 Formulating the Research Question 


This may be either a problem or an 
opportunity. At this stage you may even 
have identified symptoms rather than 
problems or opportunities. 


1a Exploration 
At this stage you review published sources and 
interview information gatekeepers to understand the true 
management dilemma, not just its symptoms. 


Using collected exploratory information, 
you word the dilemma or the correction 
of the symptom in question form, usually 
starting with “How can the organization . . .?” 


2a Exploration 
The purpose of this stage is to clarify the possible 
management actions that might be taken to solve the 
management dilemma. This stage usually involves interviews 
with information gatekeepers, brainstorming with experts, 

and other qualitative research techniques. 


Several research questions may be 
formulated at this stage. Each question is 


an alternative action that management 
might take to solve the management 
dilemma. Usually the most plausible 
action, or the one that offers the greatest 
gain using the fewest resources, is 
researched first. 


management-research question hierarchy and define the research question, the client needs to collect 
some exploratory information on: 


e What factors are contributing to the bank’s failure to achieve a stronger growth rate in deposits? 


° How well is the bank doing regarding customer satisfaction and financial condition compared to 
industry norms and competitors? 


Small-group discussions are conducted among employees and managers, and trade association data are 
acquired to compare financial and operating statistics from company annual reports and end-of-year 
division reports. From the results of these two exploratory activities, it is obvious that BankChoice’s 
operations are not as progressive as its competitors’ but it has its costs well in line. So the revised 
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>Exhibit 5-9 Types of Management Questions 


Categories General Question Sample Management Questions 


¢ What do we want to achieve? 


Should we reposition brand X as a therapeutic product 
from its current cosmetic positioning? 


Choice of Purpose or 
Choice of Objectives 


« What goals should XYZ try to achieve in its next round of 
distributor negotiations? 


Troubleshooting or Control What is our product line’s sales-to-promotion cost ratio? 
(monitoring or diagnosing 
ways an organization is 


failing to meet its goals) 


¢ How well is our marketing pro- 
gram meeting its goals? 


« Why does our department have the lowest sales-to-Web 


¢ Why is our marketing program page visit ratio? 


ing i @ 
not meeting its goals? ¢ Why does our product line have the lowest off-shelf display 


occasions in the industry? 


management question becomes What should be done to make the bank more competitive? The process 
of exploration will be critical in helping BankChoice identify its options. 

In addition to solving problems, marketers are likely to be looking for opportunities in the market- 
place. So let’s look at another case, TechByte. This company is interested in enhancing its position in 
a given technology that appears to hold potential for future growth. This interest or need might quickly 
elicit a number of questions: 


¢ How fast might this technology develop? 
e What are the likely applications of this technology? 


e What companies now possess this technology, and which ones are likely to make a major effort 
to obtain the technology? 


¢ How much will the new technology absorb in resources? 
e What are the likely payoffs? 


In the above exploration of opportunities, researchers would probably begin with specific books and 
periodicals. They would be looking only for certain aspects in this literature, such as recent develop- 
ments, predictions by informed individuals about the prospects of the technology, identification of 
those involved in the area, and accounts of successful ventures or failures by others in the field. After 
becoming familiar with the literature, researchers might seek interviews with scientists, engineers, and 
product developers who are well known in the field. They would give special attention to those who 
represent the two extremes of opinion in regard to the prospects of the technology. If possible, they 
would talk with persons having information on particularly thorny problems in development and ap- 
plication. Of course, much of the information will be confidential and competitive. However, skillful 
investigation can uncover many useful indicators. BankChoice ultimately decides to conduct a survey 
of local residents. Two hundred residents complete questionnaires, and the information collected is 
used to guide repositioning of the bank. 

In the opening MindWriter vignette, Sara takes on the task of discovering published PC indus- 
try studies on service and technical support, as well as published customer satisfaction comparisons 
among technical companies and products. Meanwhile, at MindWriter, Myra Wines is searching com- 
pany archives for prior studies on customer satisfaction. Jason has realized from reviewing Sara’s sum- 
mary of customer correspondence that Henry & Associates needs more knowledge on product design, 
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The Online Professional Community as a Source of Business Information 


When you think of online communities or networks, you likely e The last type is the professional collaboration community, 
think of consumer communities like Facebook, Twitter, or where the sharing of sensitive content requires a large 
Google+. But in the business community, these networks thrive degree of trust among members; this type of network 

as well. According to Matthew Lees, vice president and consul- is usually small, often admits members only by referral 
tant with the Patricia Seybold Group, “professional peer commu- and invitation, may contain competitors, and is often 
nities (also known as communities of practice), are built around integrated into the member's work life; this type of 

a specific topic, industry, or discipline.” Many offer both oppor- community requires diligent moderation. (In the research 
tunities for online interaction as well as face-to-face interaction. community, Next Gen Market Research (NGMR) group 
Vanessa DiMauro, a principal of Leader Networks, and devel- on Linkedin is an example.) 


oper of virtual communities for more than 15 years, indicates 


“The key requirement that surrounds the formation of any pro- 
that all professional communities are not the same. She identifies 


fessional or business-to-business community is that there needs 


UES M SES: to be a burning imperative—a driving need—for people to share 
e The information dissemination community is largely a information,” explains DiMauro. “In the creation of any profes- 
sponsored-content website where community members sional community, that need must outweigh any of the confiden- 
go to collect information, but rarely interact with each tiality issues that bring up retention of information. When dealing 
other; this type of community tends to develop sporadic with executives, it’s always a safe bet that they have a deep 
participation. (At the consumer level, WebMD is an sense of what is appropriate and inappropriate to share.” 
example.) 


www.psgroup.com; www.leadernetworks.com; 
www.tomhcanderson.com/category/ 
next-gen-market-research/. 


e The shop-talk community is interactive and represents a 
network where members can get their job-related questions 
answered; this type of community uses a moderator. 


CompleteCare’s practices, and product handling; so Jason plans a second exploratory process starting 
with an expert interview with MindWriter’s Sam Turnbull. 

An unstructured exploration allows the researcher to develop and revise the management question 
and determine what is needed to secure answers to the proposed question. 


The Research Question 


Using his or her understanding of the basic theoretical concepts, the researcher’s task is to assist the 
manager in formulating a research question that fits the need to resolve the management dilemma. A 
research question best states the objective of the business research study. It is a more specific manage- 
ment question that must be answered. It may be more than one question or just one. A business research 
process that answers this more specific question provides the manager with the information necessary 
to make the decision he or she is facing. Incorrectly defining the research question is the fundamental 
weakness in the business research process. Time and money can be wasted studying an alternative that 
won’t help the manager rectify the original dilemma. 

Meanwhile, at BankChoice the president has agreed to have the business research be guided by the 
following research question: should BankChoice position itself as a modern, progressive institution 
(with appropriate changes in services and policies) or maintain its image as the oldest, most reliable 
institution in town? 


Fine-Tuning the Research Question 


The term fine-tuning might seem to be an odd usage for research, but it creates an image that most 
researchers come to recognize. Fine-tuning the question is precisely what a skillful practitioner must 
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do after the exploration is complete. At this point, a clearer picture of the management and research 
questions begins to emerge (see Exhibit 5-10, p. 114). After the researcher does a preliminary review 
of the literature, a brief exploratory study, or both, the project begins to crystallize in one of two ways: 


1. It is apparent that the question has been answered and the process is finished. 


2. A question different from the one originally addressed has appeared. 


The research question does not have to be materially different, but it will have evolved in some fashion. 
This is not cause for discouragement. The refined research question(s) will have better focus and will 
move the business research forward with more clarity than the initially formulated question(s). 

In addition to fine-tuning the original question, the researcher should address other research 
question-related activities in this phase to enhance the direction of the project: 


1. Examine the variables to be studied. Are they satisfactorily defined? Have operational defini- 
tions been used where appropriate? 


2. Review the research questions with the intent of breaking them down into specific second- and 
third-level questions. 


3. If hypotheses (tentative explanations) are used, be certain they meet the quality tests mentioned 
in Chapter 3. 


4. Determine what evidence must be collected to answer the various questions and hypotheses. 


5. Set the scope of the study by stating what is not a part of the research question. This will estab- 
lish a boundary to separate contiguous problems from the primary objective. 


Investigative Questions 


Investigative questions represent the information that the decision maker needs to know; they are 
the questions the researcher must answer to satisfactorily arrive at a conclusion about the research 
question. To study the market, the researcher working on the BankChoice project develops two major 
investigative questions. Each question has several subquestions. These questions provide insight into 
the lack of deposit growth: 


1. What is the public’s position regarding financial services and their use? 
a. What specific financial services are used? 
b. How attractive are various services? 
c. What bank-specific and environmental factors influence a person’s use of a particular 
service? 


2. What is the bank’s competitive position? 

a. What are the geographic patterns of our customers and of our competitors’ customers? 

b. What demographic differences are revealed among our customers and those of our 
competitors? 

c. What descriptive words or phrases does the public (both customers and noncustomers) associ- 
ate with BankChoice? With BankChoice’s competitors? 

d. How aware is the public of BankChoice’s promotional efforts? 

e. What opinion does the public hold of BankChoice and its competitors? 

f. How does growth in services compare among competing institutions? 


Return again to the MindWriter situation. What does management need to know to choose among 
the different packaging specifications? As you develop your information needs, think broadly. In de- 
veloping your list of investigative questions, include: 


¢ Performance considerations (such as the relative costs of the options, the speed of packing ser- 
viced laptops, and the arrival condition of test laptops packaged with different materials). 
° Attitudinal issues (such as perceived service quality based on packaging materials used). 


¢ Behavioral issues (such as employees’ ease of use in packing with the considered materials). 
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Proposing Research for MindWriter and the Exploration Phase 


>Exhibit 5-10 Proposal for MindWriter CompleteCare Satisfaction Research 


When last we checked, Sara and Jason were preparing a proposal for Gracie Uhura, product manager at MindWriter Corporation. 

Sara decided to exclude the “executive summary” for two reasons: The proposal is short and the essentials will be contained in 
the cover letter. The proposal follows the components discussed in this chapter. It is an appropriate adaptation for an internal, small- 
scale study. The module “qualification of researcher” was not needed because MindWriter’s employee solicited the proposal; Sara 
had prejudged the researcher's qualifications. 


Repair Process Satisfaction Proposal 
MindWriter Corporation CompleteCare Program 


Problem Statement 


MindWriter Corporation has recently created a service and repair program, CompleteCare, for its portable/laptop/notebook computers. 
This program promises to provide a rapid response to customers’ service problems. 

MindWriter is currently experiencing a shortage of trained technical operators in its telephone center. The package courier, 
contracted to pick up and deliver customers’ machines to CompleteCare, has provided irregular execution. MindWriter has also 
experienced parts availability problems for some machine types. 

Recent phone logs at the call center show complaints about CompleteCare; it is unknown how representative these complaints are 
and what implications they may have for satisfaction with MindWriter products. 

Management desires information on the program’s effectiveness and its impact on customer satisfaction to determine what should 
be done to improve the CompleteCare program for MindWriter product repair and servicing. 


Research Objectives 


The purpose of this research is to discover the level of satisfaction with the CompleteCare service program. Specifically, we intend to 
identify the component and overall levels of satisfaction with CompleteCare. Components of the repair process are important targets 
for investigation because they reveal: 

(1) How customer tolerance levels for repair performance affect overall satisfaction, and 

(2) Which process components should be immediately improved to elevate the overall satisfaction of those MindWriter customers 
experiencing product failures. 

We will also discover the importance of types of product failure on customer satisfaction levels. 


Importance/Benefits 


High levels of user satisfaction translate into positive word-of-mouth product endorsements. These endorsements influence the 
purchase outcomes for (1) friends and relatives and (2) business associates. 

Critical incidents, such as product failures, have the potential to either undermine existing satisfaction levels or preserve and 
even increase the resulting levels of product satisfaction. The outcome of the episode depends on the quality of the manufacturer’s 
response. 

An extraordinary response by the manufacturer to such incidents will preserve and enhance user satisfaction levels to the point 
that direct and indirect benefits derived from such programs will justify their costs. 

This research has the potential for connecting to ongoing MindWriter customer satisfaction programs and measuring the long-term 
effects of CompleteCare (and product failure incidents) on customer satisfaction. 


Research Design 


Exploration: Qualitative. We will augment our knowledge of CompleteCare by interviewing the service manager, the call center 
manager, and the independent package company’s account executive. Based on a thorough inventory of CompleteCare’s internal 
external processes, we propose to develop an online survey. 
Questionnaire Design. An email-recruited, online, self-administered questionnaire offers the most cost-effective method for 
securing feedback on the effectiveness of CompleteCare. The introduction on the questionnaire will be a variation of MindWriter’s 
current advertising campaign. 

Some questions for this instrument will be based on the investigative questions we presented to you previously, and others will be 
drawn from the executive interviews. We anticipate a maximum of 12 questions. A new five-point expectation scale, compatible with 
your existing customer satisfaction scales, is being designed. 


(continued) 
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>Exhibit 5-10 Proposal for MindWriter CompleteCare Satisfaction Research (concluded) 


We understand that you and Mr. Malraison are interested in verbatim comments. A comments/suggestions question will be 
included. In addition, we will work out a code block that captures the call center’s reference number, model, and item(s) serviced. 
Logistics. We recommend using SurveyMonkey as the distributor of the questionnaire, for its ease of use and timeliness of data 
collection. 

Pilot Test. We will test the questionnaire with a small sample of customers. This will contain your costs. We will then revise the 
questions and forward them to our graphics designer for layout. The instrument will then be submitted to you for final approval. 
Evaluation of Nonresponse Bias. A random sample of 100 names will be secured from the list of customers who choose not to 
participate in the online survey. CompleteCare records will be used for establishing the sampling frame. Nonresponders will be 
interviewed on the telephone and their responses compared statistically to those of the responders. 


Data Analysis 


We will review and send you a weekly report listing customers who are dissatisfied (score a “1” or “2”) with any item of the 
questionnaire or who submit a negative comment. This will improve your timeliness in resolving customer complaints. 
Each month, we will provide you with a report consisting of frequencies and category percentages for each question. Visual 
displays of the data will be in bar chart/histogram form. We propose to include at least one question dealing with overall 
satisfaction (with CompleteCare and/or MindWriter). This overall question would be regressed on the individual items to determine 
each item’s importance. A performance grid will identify items needing improvement with an evaluation of priority. Other analyses 
can be prepared on a time and materials basis. 

The open-ended questions will be summarized and reported by model code. If you wish, we also can provide content analysis for 
these questions. 


Results: Deliverables 


1. Development and production of an online questionnaire. 

2. Weekly exception reports (transmitted electronically) listing customers who meet the dissatisfied customer criteria. 
3. Monthly reports as described in the data analysis section. 

4. Upload of each month’s data to a secure FTP site, by the fifth working day of each month. 


Budget 


Questionnaire Design. Development of a customized scale and online survey, incorporating the MindWriter logo using a 
customized SurveyMonkey account. MindWriter or CompleteCare logo can be positioned under the printed material at a 
nominal charge. The two-sided cards measure 4/4 by 5/. 

This allows us to print four cards per page. The opposite side will have the business reply logo, postage paid symbol, and address. 


Cost Summary 


Interviews $ 1,550.00 
Travel costs 2,500.00 
Questionnaire development 1,850.00 
Equipment/supplies 1,325.00 
Graphics design 800.00 
SurveyMonkey secure account $ 200.00 
Secure FTP site $ 300.00 
Monthly data files (each) 50.00 
Monthly reports (each) 1,850.00 

Total start-up costs $10,425.00 


Monthly run costs $ 1,030.00 
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PROPOSAL FOR MINDWRITER RESEARCH 


Two days after their Austin trip, at 1 p.m. sharp, Sara ushers 
Myra into a round Henry & Associates’ conference room. 

Inside, Jason has posted paper to the curved walls. 

Across the top of the first sheet, Sara has written, “Satis- 
faction with the service department.” Today they focus on the 
easiest task and leave the customer profile pilot study for later. 
Besides, Gracie Uhura, marketing manager for MindWriter, is 
pressed for answers on whether the CompleteCare repair pro- 
gram enhances customer satisfaction and thus brand loyalty. If 
she is responsive on the smaller project, Henry & Associates is 
sure it will get the OK for the more ambitious one. 

Jason and Myra pull two chairs in front of the first blank 
panel, at first staring in silence at its blankness. 

As Jason begins to talk, Sara summarizes his ideas on the 
panels. He has learned a lot about MindWriter. Beginning with a 
visit to the Internet and an intense search through MindWriter’s 
archives before their Austin trip, followed by the meetings in Aus- 
tin, he knows the product is sold through computer superstores 
and independent mail-order companies. He also has learned that 
MindWriter ships about 5,000 portable/laptop computers per 
month. The product is successful yet constrained by the same 
supply shortages as the rest of the industry. Personal computer 
magazines have been consulted for their annual surveys on ser- 
vice, repair, and technical support. Overall customer satisfaction 
comparisons have been obtained from published sources. 

Myra approaches the second blank panel and summa- 
rizes the information learned from the Austin trip under the 
label “CompleteCare Process.” When customers experience a 
malfunction, they call an 800 number. The call center answers 
service, support, and ordering questions. Technical representa- 
tives are trained to: 


e Take the name, phone, address, and MindWriter model 
number. 

e Listen to the customer and ask questions to detect the 
nature of the problem. 

e Attempt to resolve the problem if they can walk the 
customer through corrective steps. 


If unable to resolve the problem, the representative provides 
a return authorization code and dispatches a package courier to 
pick up the unit before 5 p.m. The unit is delivered to Austin for 
service the next morning. The CompleteCare repair facility calls 
the customer if the repair information is incomplete. The unit is 
repaired by the end of that day and picked up by the courier. The 
call center then updates its database with service record infor- 
mation. If all goes well, the customer receives the repaired unit 
by 10:00 a.m. the following morning, 48 hours after MindWriter 
received the customer’s original problem call. 

As Myra sits down, Jason begins to rough out the known 
“problems” on a third panel. There are employee shortages 
at the call center and difficulties getting the new technical 
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representatives trained. The courier is inconsistent in execut- 
ing its pickup and delivery contract. MindWriter is experiencing 
parts availability problems for some models. And, occasionally, 
units are returned to the customer either not fixed or damaged in 
some way. Jason brainstorms that the service area is not doing 
an adequate job. But Myra asserts that problems could be in the 
original packing, in handling, or even from activities related to 
taking the boxes on and off the shipping pallets. 

Their brainstorming results in a restatement of the management 
question: What should be done to improve the CompleteCare 
program (MindWriter’s program for product repairs and servic- 
ing)? After further discussion, Myra, Sara, and Jason brainstorm 
the following research and investigative questions. 


RESEARCH QUESTIONS 


1. Should the technical representative be given more inten- 
sive training? 

2. Should ABC Courier Service be replaced with an over- 
night air-transport service? 

3. Should the repair-diagnostic and repair-sequencing 
operations be modified? 

4. Should the return packaging be modified to include pre- 
molded rigid foam inserts, conforming-expanding foam 
protection, or some other configuration? 

5. Should metropolitan repair centers be established to 
complement or replace in-factory repair facilities? 


INVESTIGATIVE QUESTIONS 


1. How well is the call center helping the customers? Is it help- 
ing the customers with instructions? What percentage of 
customers’ technical problems is the center solving without 
callbacks? How long do customers wait on the phone? 

2. How good is the transportation company? Does it pick 
up and deliver the laptops responsively? How long do 
customers wait for pickup? Delivery? Are the laptops 
damaged due to package handling? What available 
packaging alternatives are cost-effective? 

3. How good is the repair group? What is the sequencing 
of the repair program, diagnostics through comple- 
tion? Is the repair complete? Are customers’ problems 
resolved? Are new repair problems emerging? Are cus- 
tomers’ repair-time expectations being met? 

4. (Do this set of questions on your own. See discussion 
question 8 at the end of this chapter.) 

5. What is the overall satisfaction with CompleteCare and 
with the MindWriter product? 


Myra now has enough information to go back to Gra- 
cie Uhura at MindWriter. In particular, Myra wants to know 
whether she and Jason have translated Gracie’s management 
question in a way that will adequately fulfill Gracie’s need for 


information. 
(continued) 
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Even though Jason had a phone interview with Sam Turnbull, 
MindWriter’s service manager, Myra and Jason may also want to 
interview the call center manager and the independent package 
company’s account executive to determine if they are on the 
right track with their investigative questions. These people will 
be able to answer some investigative questions. The rest of the 
investigative questions will need to be translated into measure- 
ment questions to ask customers. If Myra and Jason are com- 
fortable with the additional information from their interviews (and 
any additional customer letters), Henry and Associates can then 
develop a questionnaire for CompleteCare customers. 

Jason, Sara, and Myra wrap up their session discussing pre- 
liminary plans and timing. Jason wants a pilot test with a limited 
number of customers. Afterward, he will revise the questions, set 
up the logistics, and then roll out the business research program. 
Sampling will be a critical matter. If Gracie’s budget is large, they 
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can use a probability sample from the customer list that Mind- 
Writer generates every week. This will make telephone interviews 
possible. If a less expensive alternative is needed, however, they 
can propose that a questionnaire postcard survey be included 
with every laptop as it is returned to the customer. They also 
would do random sampling from the list of customers who do 
not respond. Nonresponders would be interviewed on the tele- 
phone. This way Myra and Jason can be assured of a cost- 
effective questionnaire with correction for nonresponse bias (an 
error that develops when an interviewer cannot locate or involve 
the targeted participant). 

Myra, Sara, and Jason devise a tentative schedule (see 
Exhibit 5-11) before calling to arrange the follow-up interviews. 
They want to give Gracie target dates for completion of the ex- 
ploratory phase and the instrument and pilot test, as well as a 
deadline for the first month’s results. 


>Exhibit 5-11 A Gantt Chart of the MindWriter Project 
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Measurement Questions 


Measurement questions are the actual questions that researchers use to collect data in a study. They 
could become questions on a survey or elements on an observation checklist. 

Measurement questions should be outlined by the completion of the project planning activities but 
usually await pilot testing for refinement. Two types of measurement questions are common in busi- 
ness research: 


e Predesigned, pretested questions. 


e Custom-designed questions. 


Predesigned measurement questions are questions that have been formulated and tested previously 
by other researchers, are recorded in the literature, and may be applied literally or be adapted for 
the project at hand. Some studies lend themselves to the use of these readily available measurement 
devices. Such questions provide enhanced validity and can reduce the cost of the project. Often, how- 
ever, the measurement questions must be custom tailored to the investigative questions. The resources 
for developing custom-designed measurement questions—questions formulated specifically for the 
project at hand—are the collective insights from all the activities in the business research process com- 
pleted to this point, particularly insights from exploration. Later, during the pilot testing phase of the 


research process, these custom-designed questions will be refined. 
We discuss measurement questions only briefly here; a more complete discussion begins in 


Chapter 11. 


eT 


The exploratory phase of the research process uses in- 
formation to expand understanding of the management 
dilemma, look for ways others have addressed and/or 
solved problems similar to the management dilemma or 
management question, and gather background information 
on the topic to refine the research question. Exploration of 
the problem is accomplished through familiarization with 
the available literature, interviews with experts and other 
individual depth interviews, and group discussions or some 
combination of these. Revision of the management or re- 
search questions is a desirable outcome of exploration and 
enhances the researcher’s understanding of the options 
available for developing a successful design. 


Researching secondary sources is complex and challeng- 
ing. There are two categories of sources available (external 
and internal) and three types of sources (primary, second- 
ary, and tertiary). Primary sources are original works of 
research or raw data without interpretation. Secondary 
sources are interpretations of primary data. Tertiary sources 
may be interpretations of secondary sources or, more com- 
monly, finding aids such as indexes, bibliographies, and 
Internet search engines. 


There are generally five types of information sources used 

in most literature searches, including indexes and bibliogra- 
phies, dictionaries, encyclopedias, handbooks, and directo- 
ries. Each is useful to an exploratory phase literature search 
in a variety of ways. One of the harder tasks associated with 


using secondary sources is evaluating the quality of the in- 
formation. Five factors to consider when evaluating the qual- 
ity of the source are purpose, scope, authority, audience, 
and format. 


How one structures the research question sets the direc- 
ion for the project. A management problem or opportunity 
can be formulated as a hierarchical sequence of ques- 
tions. At the base level is the management dilemma. This 
is translated into a management question and then into a 
research question—the major objective of the study. In turn, 
the research question is further expanded into investigative 
questions. These questions represent the various facets 

of the problem to be solved, and they influence research 
design, including design strategy, data collection planning, 
and sampling. At the most specific level are measurement 
questions that are answered by respondents in a Survey or 
answered about each subject in an observational study. 


Managers faced with current decisions requiring immediate 
attention often overlook internal data in a company’s data 
warehouse. Data mining refers to the process of discovering 
knowledge from databases. Data-mining technology pro- 
vides two unique capabilities to the researcher or manager: 
pattern discovery and the prediction of tends and behaviors. 
Data-mining tools perform exploratory and confirmatory 
statistical analyses to discover and validate relationships. 
Data mining involves a five-step process: sample, explore, 
modify, model, and assess. 
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Terms in Review 


1 


Explain how each of the five evaluation factors for a second- 
ary source influences its management decision-making value. 


a Purpose 
b Scope 
c Authority 
d Audience 
e Format 


Define the distinctions between primary, Secondary, and 
tertiary sources in a secondary search. 


What problems of secondary data quality must researchers 
face? How can they deal with them? 


Making Research Decisions 
4 In May 2007, TUX Co., the parent company of T.J.Maxx and 


other retailers, announced in a Securities and Exchange 
Commission filing that more than 45 million credit and debit 
card numbers had been stolen from its IT systems. The 
company had taken some measures over a period of a few 
years to protect customer data through obfuscation and en- 
cryption. But TUX didn’t apply these policies uniformly across 
its IT systems. As a result, it still had no idea of the extent of 
the damage caused by the data breach. If you were TJX, 
what data-mining research could you do to evaluate the 
safety of your customer’s personal data? 


Confronted by low sales, the president of Oaks Interna- 
tional Inc. asks a research company to study the activities 
of the customer relations department in the corporation. 
What are some of the important reasons that this research 
project may fail to make an adequate contribution to the 
solution of management problems? 


6 


You have been approached by the editor of Gentlemen’s 
Magazine to carry out a research study. The magazine has 
been unsuccessful in attracting shoe manufacturers as 
advertisers. When the sales reps tried to secure advertising 
from shoe manufacturers, they were told men’s clothing 
stores are a small and dying segment of their business. 
Since Gentlemen’s Magazine goes chiefly to men’s clothing 
stores, the manufacturers reasoned that it was, therefore, 
not a good vehicle for their advertising. The editor believes 
that a survey (via mail questionnaire) of men’s clothing 
stores in the United States will probably show that these 
stores are important outlets for men’s shoes and are not 
declining in importance as shoe outlets. He asks you to 
develop a proposal for the study and submit it to him. 
Develop the management-research question hierarchy that 
will help you to develop a specific proposal. 


Develop the management-research question hierarchy for 
a management dilemma you face at work or with an orga- 
nization to which you volunteer. 


How might you use data mining if you were a human re- 
sources officer or a supervising manager? 


Bring Research to Life 


9 


10 


Using the MindWriter postservicing packaging alternative 
as the research question, develop appropriate investigative 
questions within the question hierarchy by preparing an 
exhibit similar to Exhibit 5-8. 


Using Exhibits 5-6, 5-8, 5b-1, and 5b-2, state the research 
question and describe the search plan that Jason should 
have conducted before his brainstorming sessions with 
Myra Wines. What government sources should be included 
in Jason’s search? 
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11 Using the “uneven courier performance” problem or From the Headlines 
the “product damaged during repair” problem (see the 13 Oreo cookies turned 100 this year, and it was quite a birth- 
Closeup on page 118), develop some exploration activities day. Nabisco celebrated with a limited-edition-flavor release 
that would let Jason or Myra proceed to develop a more called Birthday Cake Oreos, with the taste of Funfetti cake 
refined research question dealing with this problem. and rainbow sprinkles within the filling center. According to 

From Concept to Practice Vanity Fair, this Oreo is a very sweet smell and taste experi- 

12 Develop the management-research question hierarchy ence. Oreos are the number one cookie, selling more than 
(Exhibits 5-6 and 5-8), citing management dilemma, manage- 70 million cookies each day. Oreo comes in more than 30 
ment question, and research question(s), for each of the flavor versions, including original, Golden, Double Stuff, and 
following: Green Tea. If you were deciding whether to introduce a new 


Oreo as part of the celebration, how would you frame the 


a The president of a home health care services firm. ; ; 
question hierarchy? 


b The vice president of investor relations for an auto 
manufacturer. 


c The retail advertising manager of a major metropolitan 
newspaper. 


d_ The chief of police in a major city. 


es 


A Gem of a Study Mastering Teacher Leadership 
ee Akron Children’s Hospital NCRCC: Teeing Up and New Strategic Direction 
Calling Up Attendance ee Ohio Lottery: Innovative Research Design Drives 
Winning 


ee Donatos: Finding the New Pizza 
Ramada Demonstrates Its Personal Best™ 


HeroBuilders.com 
State Farm: Dangerous Intersections 


Inquiring Minds Want to 
Know—NOW! ee USTA: Come Out Swinging 


* You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 
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You'll find the following appendices available from the Online Learning Center (www.mhhe.com/cooper1 2e) to supplement the 
content of this chapter: 

Appendix 5a: Bibliographic Database Searches 

Appendix 5b: Advanced Searches 
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chapter 6 


learningobjectives 
After reading this chapter, you should understand .. . 


The basic stages of research design. 
The major descriptors of research design. 
The major types of research designs. 


The relationships that exist between variables in research design and the steps for evaluating those relationships. 


6 6 Wecan forecast advances in technology, 
but we can’t really forecast human 
reactions to those advances. Maybe 
corporate researchers will invest in big 


data analytics only to discover they’re 
missing a hugely important piece: the 
story of why. 9 9 


Ron Sellers, CEO, 
GreyMatter Research & Consulting 
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“So what has you frowning at the coffee pot so early 


> 


on this beautiful morning,” greets Jason Henry, as he 
encounters his partner Sara Arens staring at the coffee 
machine. “I believe the statisticians have confirmed 
that ‘A watched pot never boils.’” 

“T’m just debating whether I need another cup before 
joining you for the debriefing on our new project,” 
shares Sara. “My mind has been running on overdrive 
since yesterday.” Sara pours herself a generous cup 
and tops off Jason’s cup as well. “Let’s move to the 
conference room. I have lots of information to share 
with you.” 

“The meeting didn’t turn out quite like we expected. 
Not just another project from a former client. What I 
faced yesterday was, well, a syndicate of businesses— 
representatives from Sun Microsystems, Best Buy, 
AT&T; even some major health institutions were 
in the group, to name just a few. In a nutshell, they 
want us to design and conduct the definitive study 
on the effectiveness of work-life balance employee 
programs,” exclaims Sara, grinning widely. 

“Work-—life balance programs—those are the ones 
that track employee performance, but not their time 
on the job, that give employees a lot more control 
over when and where they work, right?” asks Jason. 
“Weren’t you just talking about these programs being 
profiled in the article you read last week in. . .” 

i . in BusinessWeek,” interrupts Sara. “The 
exciting thing about the study is that they are so 
open on methodology. They want us to design the 
ultimate study. We will have access to data from those 
organizations that have implemented such programs— 
possibly enabling ex post facto studies. There were 
retailers, hospitals, manufacturers, high-technology 
companies present, almost every type of organization 
you could want in such a ground-breaking study. 


Several of the companies are real believers in work-life 
balance initiatives and are rolling out implementation 
of programs from division to division. As a result, 
they have some divisions in which the programs are 
not yet implemented. And these organizations cross 
continental boundaries. We’ll have to work with cross- 
cultural issues as well as workplace culture issues. 
And several other organizations within the syndicate 
are interested in implementing such work-life balance 
initiatives but have agreed not to implement programs 
until the research is in place to monitor changes in 
attitudes, perceptions, and behaviors.” 

“Ah, so that work we did in Europe and China 
last year, where the workplace cultures had such a 
significantly different impact, will pay dividends on 
this next project. So you are thinking control studies?” 
questions Jason. 

“Well, we certainly have the opportunity to set up 
such field experiments,” claimed Sara. “But initially 
I’m thinking a multiphase descriptive communication 
study, too, maybe combining qualitative research 
to get at deeper feelings and motivations for work 
environment change with surveys to generate statistical 
measures of pre- and postattitudes, but also some 
longitudinal monitoring studies—and some of these 
could be observation research. We have so many 
options that we will need some exploratory work to 
help us focus. Some of the companies had done some 
preimplementation descriptive research on morale and 
some longitudinal measures on performance changes. 
I think we should digest those reports before we make 
any decisions about design. They promised to overnight 
the reports—we may have some this morning, but we 
should have the bulk of them tomorrow.” 

“We could start our interns on a literature search 
now,” suggested Jason. 


“That’s a good idea,” comments Sara. “I’m sure the 
reports that are coming will direct us to some background 
research, but from the comments that were shared, 
it also appears that several of the initiatives started as 
brainstorms of C-level executives and some companies 
just floated the initiatives without any research at all. 
Now they are thinking that in order to justify further 
investment they need to understand better why some 
initiatives have boosted performance significantly, 
while others have had lesser success or no success—that 
gives credence to a causal/explanatory study.” 

“.. and, whether they can expect such incremental 
change to be ongoing or increase or diminish over 
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time,” analyses Jason. “They really need a predictive 
study to know whether an organization should expect 
specific performance changes with the implementation 
of unique initiatives in different workplace cultures.” 

“So when do they want us to present our recom- 
mended design?” 

“T’ve set up a meeting for that presentation in four 
weeks. Your schedule was free on both Thursday and 
Friday that week,” shared Sara. “They want us both 
there along with representatives of any suppliers that 
we plan to use.” 


The topics covered by the term research design are wide-ranging, as depicted in Exhibit 6-1. This 
chapter introduces a classification of research designs and provides an overview of the most important 
design types (exploratory, descriptive, and causal). We refer you to subsequent chapters for a more 
thorough coverage of the unique features of qualitative studies, observational studies, surveys, and 
experiments. Our objective here is not for you to acquire the details of research design in one reading 


Exhibit 6-1 Design in the Research Process 


Proposal 
Approved 


Research Design 


Strategy Chapters 6-14 
(type, purpose, time frame, scope, environment) 
Data Collection Sampling 
Design Design 
Data Collection & Preparation Chapter 15 
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but for you to understand its scope and to get a glimpse of the available options for tailoring a design 
to an organization’s particular research needs. 

There are many definitions of research design, but no single definition imparts the full range of 
important aspects. 


e Research design constitutes the blueprint for the collection, measurement, and analysis of data. 


¢ Research design aids the researcher in the allocation of limited resources by posing crucial 
choices in methodology.! 


e Research design is the plan and structure of investigation so conceived as to obtain answers 
to research questions. The plan is the overall scheme or program of the research. It includes 
an outline of what the investigator will do from writing hypotheses and their operational 
implications to the final analysis of data.’ 


e Research design expresses both the structure of the research problem—the frame-work, 
organization, or configuration of the relationships among variables of a study—and the plan of 
investigation used to obtain empirical evidence on those relationships.* 


These definitions differ in detail, but together they give the essentials of research design: 


e An activity- and time-based plan. 

e A plan always based on the research question. 

¢ A guide for selecting sources and types of information. 

e A framework for specifying the relationships among the study’s variables. 

e A procedural outline for every research activity. 
At its core, research is a project and project management tools such as critical path method (CPM) 
can be used to depict sequential and simultaneous steps and estimate scheduling and timetables for 
each activity or phase of the research, as is done in Exhibit 6-2. The pathway from start to end that 
takes the longest time to complete is called the critical path. Any delay in an activity along this path 
will delay the end of the entire project. We introduced you to an alternative scheduling tool, the Gantt 


chart, in Chapter 5 (see Exhibit 5-11, p. 117). Before you develop a schedule, however, you need to 
know precisely what research you plan to do. 


>Exhibit 6-2 CPM Schedule of Research Design. 


Final check of background records 


Conduct 
Client exploratory 
interviews interviews 


Complete Field Code data 
questionnaire interviewing | and analyze 


Write 
report 


Literature Develop 


: Complete executive 
review proposal P 


3 : : Interview 
questionnaire 


executives 


Arrange executive 
appointments 


> Milestones: Critical Path: 
S-1-3-4-7-8-9-E 


3 Proposal approval 
7 Interviews completed Time to Completion: 
9 Final report completed 40 working days 
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>Exhibit 6-3 Descriptors of Research Design 


Category Options 


The method of data collection Monitoring 


Communication study 


The purpose of the study e Reporting 
e Descriptive 
e Causal 
e Explanatory 


e Predictive 


The topical scope—breadth and depth—of the study Case 
Statistical study 


The participants’ perceptions of research activity e Actual routine 


e Modified routine 


Classification of Designs 


Early in any research study, one faces the task of selecting the specific design to use. A number of 
different design dimensions exist, but, unfortunately, no simple classification system defines all the 
variations that must be considered. Exhibit 6-3 groups research design issues using eight differ- 
ent descriptors.* A brief discussion of these descriptors illustrates their nature and contribution to 
research. 


Degree of Research Question Crystallization 


A study may be viewed as exploratory or formal. The essential distinctions between these two options 
are the degree of structure and the immediate objective of the study. Exploratory studies tend toward 
loose structures with the objective of discovering future research tasks. The immediate purpose of 
exploration is usually to develop hypotheses or questions for further research. The formal study begins 
where the exploration leaves off—it begins with a hypothesis or research question and involves precise 
procedures and data source specifications. The goal of a formal research design is to test the hypotheses 
or answer the research questions posed. 

The exploratory—formal study dichotomy is less precise than some other classifications. All stud- 
ies have elements of exploration in them, and few studies are completely uncharted. The sequence 
discussed in Chapter 4 (see Exhibit 4-1 and the model on the inside front cover) suggests that more 
formalized studies contain at least an element of exploration before the final choice of design. More 
detailed consideration of exploratory research is found later in this chapter. 
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Method of Data Collection 


This classification distinguishes between monitoring and communication processes. We use the term 
communication to contrast with monitoring because collecting data by questioning encompasses more 
than the survey method. Monitoring includes studies in which the researcher inspects the activities 
of a subject or the nature of some material without attempting to elicit responses from anyone. Traf- 
fic counts at an intersection, license plates recorded in a restaurant parking lot, a search of the library 
collection, an observation of the actions of a group of decision makers, the State Farm Dangerous 
Intersection Study—all are examples of monitoring. In each case the researcher notes and records 
the information available from observations. Monitoring for MindWriter might include “following” a 
computer through the repair process, documenting each activity or interaction between CompleteCare 
and call center employees and the damaged laptop. 

In the communication study, the researcher questions the subjects and collects their responses 
by personal or impersonal means. The collected data may result from (1) interview or telephone 
conversations, (2) self-administered or self-reported instruments sent through the mail, left in con- 
venient locations, or transmitted electronically or by other means, or (3) instruments presented be- 
fore and/or after a treatment or stimulus condition in an experiment. Sara and Jason propose a 
communication study, using an online survey among owners of laptops returned after CompleteCare 
servicing. 


Researcher Control of Variables 


In terms of the researcher’s ability to manipulate variables, we differentiate between experimental and 
ex post facto designs. In an experiment, the researcher attempts to control and/or manipulate the vari- 
ables in the study. It is enough that we can cause variables to be changed or held constant in keeping 
with our research objectives. Experimental design is appropriate when one wishes to discover whether 
certain variables produce effects in other variables. Experimentation provides the most powerful sup- 
port possible for a hypothesis of causation. 

With an ex post facto design, investigators have no control over the variables in the sense of being 
able to manipulate them. They can only report what has happened or what is happening. It is impor- 
tant that the researchers using this design not influence the variables; to do so introduces bias. The 
researcher is limited to holding factors constant by judicious selection of subjects according to strict 
sampling procedures and by statistical manipulation of findings. MindWriter is planning an ex post 
facto design. 


The Purpose of the Study 


The essential difference between the various studies in this grouping—reporting, descriptive, and 
causal-explanatory or causal-predictive—tlies in their objectives. A reporting study provides a sum- 
mation of data, often recasting data to achieve a deeper understanding or to generate statistics for com- 
parison. In a study of crime, for example, a reporting study might tally the number of employee thefts 
that take place in shopping malls versus free-standing stores. If the research is concerned with finding 
out who, what, where, when, or how much, then the study is descriptive. Descriptive research on em- 
ployee theft would measure the types of theft committed (clothing vs. electronics vs. housewares), how 
often, when (time of year, time of day, day of week), where (receiving dock, stockroom, sales floor), 
and by whom (gender, age, years of service, departmental assignment). If a study is concerned with 
learning why—that is, how one variable produces changes in another—it is causal-explanatory. In a 
causal-explanatory study, we try to explain relationships among variables—for instance, why the crime 
rate is higher in mall A than in mall B or why male employees steal more than female employees. A 
causal-predictive study attempts to predict an effect on one variable by manipulating another variable 
while holding all other variables constant. In our crime example, researchers using a causal-predictive 
study might be interested in whether installation of video surveillance cameras on the receiving dock 
and in stockrooms would reduce employee theft in mall stores. At the outset, the MindWriter project is 
descriptive, although subsequent studies might be causal. 
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The Time Dimension 


Cross-sectional studies are carried out once and represent a snapshot of one point in time. Longitu- 
dinal studies are repeated over an extended period. The advantage of a longitudinal study is that it can 
track changes over time. Jason and Sara’s proposal describes a longitudinal study, with satisfaction 
measurements taken continuously over several months and reported monthly. 

In longitudinal studies of the panel variety, the researcher may study the same people over time. In 
marketing, panels are set up to report consumption data on a variety of products. These data, collected 
from national samples, provide a major data bank on relative market share, consumer response to new 
products, and new promotional methods. Other longitudinal studies, such as cohort groups, use dif- 
ferent subjects for each sequenced measurement. The service industry might have looked at the needs 
of aging baby boomers by sampling 40- to 45-year-olds in 1990 and 50- to 55-year-olds in 2000. Al- 
though each sample would be different, the population of 1945 to 1950 cohort survivors would remain 
the same. 

Some types of information once collected cannot be collected a second time from the same person 
without the risk of bias. The study of public awareness of an advertising campaign over a six-month 
period would require different samples for each measurement. 

While longitudinal research is important, the constraints of budget and time impose the need for 
cross-sectional analysis. Some benefits of a longitudinal study can be revealed in a cross-sectional 
study by adroit questioning about past attitudes, history, and future expectations. Responses to these 
kinds of questions should be interpreted with care, however. 


The Topical Scope 


The statistical study differs from the case study in several ways. Statistical studies are designed for 
breadth rather than depth. They attempt to capture a population’s characteristics by making inferences 
from a sample’s characteristics. Hypotheses are tested quantitatively. Generalizations about findings 
are presented based on the representativeness of the sample and the validity of the design. MindWriter 
plans a statistical study. 

Case studies place more emphasis on a full contextual analysis of fewer events or conditions and 
their interrelations. Although hypotheses are often used, the reliance on qualitative data makes support 
or rejection more difficult. An emphasis on detail provides valuable insight for problem solving, evalu- 
ation, and strategy. This detail is secured from multiple sources of information. It allows evidence to be 
verified and avoids missing data. Remember the proposed monitoring study for MindWriter? If Mind- 
Writer tracked one or more laptops, this could serve as a case study of the CompleteCare program. 

Although case studies have been maligned as “scientifically worthless” because they do not meet 
minimal design requirements for comparison,° they have a significant scientific role. It is known that 
“important scientific propositions have the form of universals, and a universal can be falsified by a 
single counterinstance.’ Thus, a single, well-designed case study can provide a major challenge to a 
theory and provide a source of new hypotheses and constructs simultaneously. Discovering new hy- 
potheses to correct postservice complaints would be the major advantage of tracking a given number 
of damaged MindWriter laptops through the case study design. 


The Research Environment 


Designs also differ as to whether they occur under actual environmental conditions (field conditions) 
or under staged or manipulated conditions (laboratory conditions). 

To simulate is to replicate the essence of a system or process. Simulations are increasingly used 
in research, especially in operations research. The major characteristics of various conditions and re- 
lationships in actual situations are often represented in mathematical models. Role-playing and other 
behavioral activities may also be viewed as simulations. A simulation for MindWriter might involve 
an arbitrarily damaged laptop being tracked through the call center and the CompleteCare program, 
monitoring results at each workstation. Another popularly used simulation is the retail service study 
involving “mystery shoppers.” 
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Participants’ Perceptual Awareness 


The usefulness of a design may be reduced due to a participant’s perceptual awareness when people 
in a disguised study perceive that research is being conducted. Participants’ perceptual awareness influ- 
ences the outcomes of the research in subtle ways or more dramatically as we learned from the pivotal 
Hawthorne studies of the late 1920s. Although there is no widespread evidence of attempts by partici- 
pants or respondents to please researchers through successful hypothesis guessing or evidence of the 
prevalence of sabotage, when participants believe that something out of the ordinary is happening, they 
may behave less naturally. There are three levels of perception: 


1. Participants perceive no deviations from everyday routines. 
2. Participants perceive deviations, but as unrelated to the researcher. 


3. Participants perceive deviations as researcher-induced.’ 


The “mystery shopper” scenario is the perfect example of the final level of perceptual awareness 
noted in the preceding list. If a retail sales associate knows she is being observed and evaluated—with 
consequences in future compensation, scheduling, or work assignment—she is likely to change her 
performance. In all research environments and control situations, researchers need to be vigilant to 
effects that may alter their conclusions. Participants’ perceptions serve as a reminder to classify one’s 
study by type, to examine validation strengths and weaknesses, and to be prepared to qualify results 
accordingly. 


> Exploratory Studies 


Exploration is particularly useful when researchers lack a clear idea of the problems they will meet 
during the study. Through exploration researchers develop concepts more clearly, establish priorities, 
develop operational definitions, and improve the final research design. Exploration may also save time 
and money. If the problem is not as important as first thought, more formal studies can be canceled. 

Exploration serves other purposes as well. The area of investigation may be so new or so vague 
that a researcher needs to do an exploration just to learn something about the dilemma facing 
the manager. Important variables may not be known or thoroughly defined. Hypotheses for the 
research may be needed. Also, the researcher may explore to be sure it is practical to do a for- 
mal study in the area. A federal government agency, the Office of Industry Analysis, proposed 
that research be done on how executives in a given industry made decisions about raw material 
purchases. Questions were planned asking how (and at what price spreads) one raw material was 
substituted for another in certain manufactured products. An exploration to discover if industry 
executives would divulge adequate information about their decision making on this topic was es- 
sential for the study’s success. 

Despite its obvious value, researchers and managers alike give exploration less attention than it 
deserves. There are strong pressures for quick answers. Moreover, exploration is sometimes linked to 
old biases about qualitative research: subjectiveness, nonrepresentativeness, and nonsystematic design. 
More realistically, exploration saves time and money and should not be slighted. 


Qualitative Techniques 


The objectives of exploration may be accomplished with different techniques. Both qualitative and 
quantitative techniques are applicable, although exploration relies more heavily on qualitative 
techniques (also see Chapter 7). One author creates a verbal picture to differentiate the two: 


Quality is the essential character or nature of something; quantity is the amount. Quality is the what; quantity the how 
much. Qualitative refers to the meaning, the definition or analogy or model or metaphor characterizing something, while 
quantitative assumes the meaning and refers to a measure of it... . The difference lies in Steinbeck’s [1941] descrip- 
tion of the Mexican Sierra, a fish from the Sea of Cortez. One can count the spines on the dorsal fin of a pickled Sierra, 
17 plus 15 plus 9. “But,” says Steinbeck, “if the Sierra strikes hard on the line so that our hands are burned, if the fish 
sounds and nearly escapes and finally comes in over the rail, his colors pulsing and his tail beating the air, a whole new 
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relational externality has come into being.” Qualitative research would define the being of fishing, the ambiance of a 
city, the mood of a citizen, or the unifying tradition of a group.® 


When we consider the scope of qualitative research, several approaches are adaptable for explor- 
atory investigations of management questions: 


e Individual depth interviews (usually conversational rather than structured). 
¢ Participant observation (to perceive firsthand what participants in the setting experience). 
e Films, photographs, and videotape (to capture the life of the group under study). 


e Projective techniques and psychological testing (such as a Thematic Apperception Test, projec- 
tive measures, games, or role-playing). 


° Case studies (for an in-depth contextual analysis of a few events or conditions). 


¢ Street ethnography (to discover how a cultural subgroup describes and structures its world at the 
street level). 


e Elite or expert interviewing (for information from influential or well-informed people in an 
organization or community). 


¢ Document analysis (to evaluate historical or contemporary confidential or public records, 
reports, government documents, and opinions). 


e Proxemics and kinesics (to study the use of space and body-motion communication, respectively).’ 


When these approaches are combined, four exploratory techniques emerge with wide applicability 
for the management researcher: 


1. Secondary data analysis. 
2. Experience surveys. 
3. Focus groups. 


4. Two-stage designs. 


Secondary Data Analysis 


The first step in an exploratory study is a search of the secondary literature. Studies made by others for 
their own purposes represent secondary data. It is inefficient to discover anew through the collection 
of primary data or original research what has already been done and reported at a level sufficient for 
management to make a decision. 

Within secondary data exploration (refer to Chapter 5), a researcher should start first with an or- 
ganization’s own data archives. Reports of prior research studies often reveal an extensive amount of 
historical data or decision-making patterns. By reviewing prior studies, you can identify methodologies 
that proved successful and unsuccessful. Solutions that didn’t receive attention in the past due to dif- 
ferent environmental circumstances are revealed as potential subjects for further study. The researcher 
needs to avoid duplication in instances when prior collected data can provide sufficient information for 
resolving the current decision-making dilemma. While MindWriter’s CompleteCare program is newly 
introduced, it is likely that one or more studies of the previous servicing practices and policies revealed 
customer attitudes on which MindWriter based the design of the current program. 

The second source of secondary data is published documents prepared by authors outside the spon- 
sor organization. There are tens of thousands of periodicals and hundreds of thousands of books on all 
aspects of business. Data from secondary sources help us decide what needs to be done and can be a 
rich source of hypotheses. Special catalogs, subject guides, and electronic indexes—available in most 
libraries—will help in this search. In many cases you can conduct a secondary search from your home 
or office using a computer, an online service, or an Internet gateway. Regarding MindWriter, thousands 
of articles have been written on customer service, and an Internet search using the keyword customer 
service reveals tens of thousands of hits. 

If one is creative, a search of secondary sources will supply excellent background information as 
well as many good leads. Yet if we confine the investigation to obvious subjects in bibliographic 
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McDonald’s launched a long-term qualitative research project, 
its nationwide “listening tour,” in part as a corporate response 
to the public outcry that its products contribute to childhood 
obesity. In this effort, McDonald’s leadership and key decision 
makers are gathering information firsthand on the company’s 
products, nutritional messaging, and sustainability initiatives. 
The listening tour is expected to continue at the regional and 
local level once the corporate program is completed. 

The initiative grew from a question: “How do we get our 
leaders exposed to some of the conversations that are happen- 
ing out there, and how do we get those conversations and the 
people who are having them to have that unique contact with 
our leaders?” shared Ashlee Yingling, manager of U.S. media 
relations and listening tour programming at McDonald's Corp. 

“The entire premise was based on a two-way dialog,” ex- 
plained Yingling. With each specific session, McDonald’s had a 
particular focus. Rather than a free-flowing conversation, each 
session sticks to a specific sustainability topic, like nutrition and 
well-being, sustainable supply chain, environmental responsibil- 
ity, employee experience, and community. Unlike a focus group, 
each session of the listening tour had the participants and the ex- 
ecutives in the same room. Participants speak one-on-one with 
McDonald's representatives in a casual, informal setting. “It gives 
the ability for the audience to connect with them and vice versa, 
spend some time before and after talking,” described Yingling. 

Some of the stops on the tour included conversations with fe- 
male bloggers, social-media sawy parents, PTA members and 
educators, professional dieticians, as well as special interest groups 
like the League of United Latin American Citizens, in locations from 
coast to coast. For staff topics, McDonald’s used internal audiences. www.mcdonalds.com; www.cdc.gov 


McDonald’s has learned they need to be more transparent, 
and do a better job of communicating. “There’s a lot of things 
we’re doing as a brand that are resonating with customers, but 
there is a lot of room for improvement, too,” stated Yingling. “We 
need to do a better job telling our story.” 

The Center for Disease Control estimates that 17 percent of 
children and adolescents meet obesity standards. To be defined as 
obese, a child or adolescent must have a body mass index (BMI) at 
or above the 95th percentile for children of the same age and sex. 


sources, we will often miss much of the best information. Suppose the Copper Industry Association 
is interested in estimating the outlook for the copper industry over the next 10 years. We could search 
through the literature under the headings “copper production” and “copper consumption.” However, a 
search restricted to these two topics would miss more than it finds. When a creative search of the cop- 
per industry is undertaken, useful information turns up under the following reference headings: mines 
and minerals; nonferrous metals; forecasting; planning; econometrics; consuming industries such as 
automotive and communications; countries where copper is produced, such as Chile; and companies 
prominent in the industry, such as Anaconda and Kennecott. 


Experience Survey 


While published data are a valuable resource, it is seldom that more than a fraction of the existing 
knowledge in a field is put into writing. A significant portion of what is known on a topic, while in 
writing, may be proprietary to a given organization and thus unavailable to an outside searcher. Also, 
internal data archives are rarely well organized, making secondary sources, even when known, difficult 
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to locate. Thus, we will profit by seeking information from persons experienced in the area of study, 
tapping into their collective memories and experiences. 

When we interview persons in an experience survey, we should seek their ideas about important 
issues or aspects of the subject and discover what is important across the subject’s range of knowledge. 
The investigative format we use should be flexible enough so that we can explore various avenues that 
emerge during the interview. 


e What is being done? 

e What has been tried in the past without success? With success? 

¢ How have things changed? 

e What are the change-producing elements of the situation? 

¢ Who is involved in decisions and what role does each person play? 
e What problem areas and barriers can be seen? 

e What are the costs of the processes under study? 

e Whom can we count on to assist and/or participate in the research? 
e What are the priority areas? 


The product of such questioning may be a new hypothesis, the discarding of an old one, or informa- 
tion about the practicality of doing the study. Probing may show whether certain facilities are available, 
what factors need to be controlled and how, and who will cooperate in the study. 

Discovery is more easily carried out if the researcher can analyze cases that provide special insight. 
Typical of exploration, we are less interested in getting a representative cross section than in getting 
information from sources that might be insightful. Assume we study StarAuto’s automobile assembly 
plant. It has a history of declining productivity, increasing costs, and growing numbers of quality 
defects. People who might provide insightful information include: 


e Newcomers to the scene—employees or personnel who may have been recently transferred to 
this plant from similar plants. 


¢ Marginal or peripheral individuals—persons whose jobs place them on the margin between 
contending groups. First-line supervisors and lead workers are often neither management nor 
worker but something in between. 


¢ Individuals in transition—recently promoted employees who have been transferred to new 
departments. 


¢ Deviants and isolates—those in a given group who hold a different position from the majority, 
as well as workers who are happy with the present situation, highly productive departments and 
workers, and loners of one sort or another. 


e “Pure” cases or cases that show extreme examples of the conditions under study—the most un- 
productive departments, the most antagonistic workers, and so forth. 


° Those who fit well and those who do not—the workers who are well established in their orga- 
nizations versus those who are not, those executives who fully reflect management views and 
those who do not. 


¢ Those who represent different positions in the system—unskilled workers, assemblers, superin- 
tendents, and so forth.!° 


Jason and Sara plan to interview three managers during the early phase of their research for Mind- 
Writer: the managers of (1) the service facility, (2) the call center, and (3) the contract courier service. 
Their emphasis should be not only on finding out what has been done in the past but also on discover- 
ing the parameters of feasible change. They might want to expand their interviews to include long-term 
employees of the various departments, as their views are likely to be different from those of their manag- 
ers. Because postpurchase service problems might be directly related to product design, expanding their 
experience survey to individuals associated with engineering and production should also be considered. 


Focus Groups 


Focus groups became widely used in research during 
the 1980s and are used for increasingly diverse research 
applications today.'' A focus group is a group of people 
(typically 6 to 10 participants), led by a trained modera- 
tor, who meet for 90 minutes to 2 hours. The facilitator 
or moderator uses group dynamics principles to focus or 
guide the group in an exchange of ideas, feelings, and 
experiences on a specific topic. 

One topical objective of a focus group might be a new 
product or product concept, a new employee motivation 
program, or improved production-line organization. The 
basic output of the session is a list of ideas and behavioral 
observations, with recommendations by the moderator. 
These ideas and observations are often used for later 
quantitative testing. In exploratory research, the quali- 
tative data that focus groups produce may be used for 
enriching all levels of research questions and hypotheses 
and comparing the effectiveness of design options. The 
most common application of focus group research con- 
tinues to be in the consumer arena. However, corpora- 
tions are using focus group results for diverse exploratory 
applications. 

MindWriter could use focus groups involving employ- 
ees (of the call center and service departments) to deter- 
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How do you get inside the mind of a potential respondent? Specialty 
research firms compile sample frames of millions of people with di- 
verse interests who are willing to share their opinions during focus 
groups or through surveys. You might need to research dads, who 
work as professionals, play golf, and secretly love chick-flick movies. 
A great panel company can deliver such individuals, as well as many 


others with unique characteristics. 


mine changes and provide an analysis of change ideas. It 
may want focus groups with customers (both dissatisfied 
and satisfied) to uncover what has occurred in their different experiences. In another application, when 
a large title insurance company was developing a computerized help system, it ran focus groups with 
its branch office administrators to discover their preferences for distributing files on the company’s 
intranet (a company’s proprietary network—behind a security “firewall” that limits access to autho- 
rized users only). In other cases, a small college used focus groups to develop a plan to attract more 
freshmen applications, and a blood center used a focus group to improve blood donations.’* 


Two-Stage Design 


A useful way to design a research study is as a two-stage design. With this approach, exploration becomes a 
separate first stage with limited objectives: (1) clearly defining the research question and (2) developing the 
research design. 

In arguing for a two-stage approach, we recognize that much about the problem is not known but 
should be known before effort and resources are committed. In these circumstances, one is operating 
in unknown areas, where it is difficult to predict the problems and costs of the study. Proposals that 
acknowledge the practicality of this approach are particularly useful when the research budget is inflex- 
ible. A limited exploration for a specific, modest cost carries little risk for both sponsor and researcher 
and often uncovers information that reduces the total research cost. 

An exploratory study is finished when the researchers have achieved the following: 


e Established the major dimensions of the research task. 


¢ Defined a set of subsidiary investigative questions that can be used as guides to a detailed 
research design. 


e Developed several hypotheses about possible causes of a management dilemma. 


e Learned that certain other hypotheses are such remote possibilities that they can be safely 
ignored in any subsequent study. 


¢ Concluded additional research is not needed or is not feasible. 
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Wildcat Survey—Problem or Opportunity? 


A wildcat survey is executed when business managers bypass 
their internal research departments and, using online survey soft- 
ware, launch their own survey. This is a growing phenomenon, 
according to Ruth Stanat, president and CEO of SIS Interna- 
tional Research, and is most frequently done “to control costs 
and gain immediate feedback.” 

But there are drawbacks, most notably loss of control of 
internal sample panels; duplication of contact and sample fa- 
tigue; and insufficient training in question development and data 
analysis, leading to poor data quality and weak data insights. “If 
[sample respondents] feel bombarded by our surveys,” shared 
Sharon Starr, director of market research for IPC Inc., “they will 
start to ignore surveys at best, or resent the company for wast- 
ing their time at worst. The company’s image will suffer if the 
company looks disunified and out of control.” 

The availability of seemingly simple survey solutions from 
irms like Zoomerang and SurveyMonkey has magnified the 
problem. Jeffrey C. Adler, president of Centrac DC Marketing 
Research, likens it to trying to fix a broken switch on his own 
urnace. “I could have gone to Home Depot and purchased the 
switch myself, theoretically saving a lot of money. However, | 
was smart enough to recognize that changing the switch was 
not my area of expertise. How big would the savings have been 
if | wired the thing myself and caused damage to the furnace or 
burned down the house?” Adler argues that when end-users 


> Descriptive Studies 


In contrast to exploratory studies, more formalized studies are typically structured with clearly stated 


work through research departments or specialists, they are pay- 
ing for expertise in using the tools. He laments that managers 
recognize this easily with an electrician or a surgeon; but fail to 
recognize this with research specialists. 

Some in the industry think research specialists have, in part, 
created the problem by not providing Knowledge and insights to 
the manager about the practice of research itself. By not edu- 
cating that manager about how research is done, that manager 
is less able to distinguish greater from lesser quality research. 
If research buyers or sponsors understood the real challenges 
of research in a specific project, it is argued, they would have a 
greater appreciation for the art and science of research. 

“There is an opportunity for research departments to extend 
an olive branch to other parts of the business by providing best 
practices, templates, and even resources for self-executed proj- 
ects,” suggests Josh Mendelsohn, vice president of Chadwick 
Martin Bailey Inc. “I certainly wish everyone had the time/budget 
to do full-fledged research projects in every case, but sometimes 
the business need doesn’t justify the cost/time it takes to do 
something the right way. By providing assistance, the validity is- 
sues are likely to go away and redundancies are going to be less. 
And research [divisions] then become more of a business part- 
ner to be consulted than an internal vendor to be battled with.” 


www.sismarketresearch.com; www.ipc.org; 
www.centracdc.com; www.cmbinfo.com 


hypotheses or investigative questions. Formal studies serve a variety of research objectives: 


1. Descriptions of phenomena or characteristics associated with a subject population 
(the who, what, when, where, and how of a topic). 


2. Estimates of the proportions of a population that have these characteristics. 


3. Discovery of associations among different variables. 


The third study objective is sometimes labeled a correlational study, a subset of descriptive studies. 
A descriptive study may be simple or complex; it may be done in many settings. Whatever the form, a 
descriptive study can be just as demanding of research skills as the causal study, and we should insist 
on the same high standards for design and execution. 

The simplest descriptive study concerns a univariate question or hypothesis in which we ask about, 
or state something about, the size, form, distribution, or existence of a variable. In the account analy- 
sis at BankChoice (introduced in Chapters 4 and 5) we might be interested in developing a profile of 
savers. We first may want to locate them in relation to the main office. The question might be, What 
percentage of the savers live within a two-mile radius of the office? Using the hypothesis format, we 
might predict, 60 percent or more of the savers live within a two-mile radius of the office. 
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We may also be interested in securing information about other variables, such as the relative size of ac- 
counts, the number of accounts for minors, the number of accounts opened within the last six months, and 
the amount of activity (number of deposits and withdrawals per year) in accounts. Data on each of these 
variables, by themselves, may have value for management decisions. Bivariate relationships between these 
or other variables may be of even greater interest. Cross-tabulations between the distance from the account 
owner’s residence or employment to the branch and account activity may suggest that differential rates of ac- 
tivity are related to account owner location. A cross-tabulation of account size and gender of account owner 
may also show interrelation. Such findings do not imply a causal relationship. In fact, our task is to determine 
if the variables are independent (or unrelated) and if they are not, then to determine the strength or magnitude 
of the relationship. Neither procedure tells us which variable is the cause. For example, we might be able to 
conclude that gender and account size are related but not that gender is a causal factor in account size. 

Descriptive studies are often much more complex than this example. One study of savers began as de- 
scribed and then went into much greater depth. Part of the study included an observation of account records 
that revealed a concentration of nearby savers. Their accounts were typically larger and more active than those 
whose owners lived at a distance. A sample survey of savers provided information on stages in the family life 
cycle, attitudes toward savings, family income levels, and other matters. Correlation of this information with 
known savings data showed that women owned larger accounts. Further investigation suggested that women 
with larger accounts were often widowed or working single women who were older than the average account 
holder. Information about their attitudes and savings practices led to new business strategies at the bank. 
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Some evidence collected led to causal questions. The correlation between nearness to the office 
and the probability of having an account at the office suggested the question, Why would people 
who live far from the office have an account there? In this type of question a hypothesis makes 
its greatest contribution by pointing out directions that the research might follow. It might be 
hypothesized that: 


1. Distant savers (operationally defined as those with addresses more than two miles from the of- 
fice) have accounts at the office because they once lived near the office; they were “near” when 
the account decision was made. 


2. Distant savers actually live near the office, but the address on the account is outside the 2-mile 
radius; they are “near,” but the records do not show this. 


3. Distant savers work near the office; they are “near” by virtue of their work location. 


4. Distant savers are not normally near the office but responded to a promotion that encouraged 
savers to bank via computer; this is another form of “nearness” in which this concept is trans- 
formed into one of “convenience.” 


When these hypotheses were tested, it was learned that a substantial portion of the distant savers 
could be accounted for by hypotheses 1 and 3. The conclusion: Location was closely related to saving 
at a given association. The determination of cause is not so simple, however, and these findings still fall 
within the definition of a descriptive study. 

MindWriter could benefit from a descriptive study that profiles satisfied service customers versus 
dissatisfied ones. Service customer characteristics could then be matched with specific types of service 
problems, which could lead to identifying changes in product design or customer service policies. 


> Causal Studies 


The correlation between location and probability of account holding at BankChoice might look like 
strong evidence to many, but the researcher with scientific training will argue that correlation (simulta- 
neous occurrence) is not causation. The essence of the disagreement seems to lie in the concept of cause. 

The essential element of causation is that A “produces” B or A “forces” B to occur. Empirically, we 
can never demonstrate an A-B causality with certainty. The reason is that we do not “demonstrate” such 
causal linkages deductively. Unlike deductive conclusions, empirical conclusions are inferences— 
inductive conclusions. As such, they are statements of the probability that A “produces” B based on 
what we observe and measure. 

In Chapter 3 we discussed the example of sales failing to increase following a promotion. Having 
ruled out other causes for the flat sales, we were left with one inference that was probably but not 
certainly the cause: a poorly executed promotion. 

Meeting the ideal standard of causation requires that one variable always causes another and no 
other variable has the same causal effect. The method of agreement, proposed by John Stuart Mill in the 
19th century, states, “When two or more cases of a given phenomenon have one and only one condition 
in common, then that condition may be regarded as the cause (or effect) of the phenomenon.”!* Thus, if 
we can find Z and only Z in every case where we find C, and no others (A, B, D, or E) are found with Z, 
then we can conclude that C and Z are causally related. Exhibit 6-4 illustrates this method. 

An example of the method of agreement might be the problem of occasional high absenteeism on 
Mondays in a factory. A study of two groups with high absenteeism (No. | and No. 2 in Exhibit 6-4) 
shows no common job, department, demographic, or personal characteristics (A, B, D, and E). How- 
ever, membership in a camping club (C) is common across both groups. The conclusion is that club 
membership is associated with high absenteeism (Z). 

The method of agreement helps rule out some variables as irrelevant. In Exhibit 6-3, A, B, D, and E 
are unlikely to be causes of Z. However, there is an implicit assumption that there are no variables to 
consider other than A, B, C, D, and E. One can never accept this supposition with certainty because the 
number of potential variables is infinite. In addition, while C may be the cause, it may instead function 
only in the presence of some other variable not included. 
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>Exhibit 6-4 Mill's Method of Agreement 


Descriptive Factors Outcome 
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The negative canon of agreement states that where the absence of C is associated with the absence 
of Z, there is evidence of a causal relationship between C and Z. Together with the method of agree- 
ment, this forms the basis for the method of difference: “If there are two or more cases, and in one of 
them observation Z can be made, while in the other it cannot; and if variable C occurs when observation 
Z is made, and does not occur when observation Z is not made; then it can be asserted that there is a 
causal relationship between C and Z.’”* 

Using our MindWriter example, if Jason and Sara were to discover that a particular servicing prob- 
lem repeatedly occurred only when a single employee was involved in the servicing of customers’ 
laptops and never when that employee was absent, an assumption of causation might be made. The 
method of difference is illustrated in Exhibit 6-5. Although these methods neither ensure discovery of 
all relevant variables nor provide certain proof of causation, they help advance our understanding of 
causality by eliminating inadequate causal arguments.'° 

While no one can ever be certain that variable A causes variable B to occur, one can gather some evidence 
that increases the belief that A leads to B. In testing causal hypotheses, we seek three types of evidence: 


1. Covariation between A and B. 
° Do we find that A and B occur together in the way hypothesized (symmetrical relationship)? 
e When A does not occur, is there also an absence of B? 
e When there is more or less of A, does one also find more or less of B? 


2. Time order of events moving in the hypothesized direction. 
¢ Does A occur before B? 


3. No other possible causes of B. 
e Can one determine that C, D, and E do not covary with B in a way that suggests possible 
causal connections? 


In addition to these three conditions, successful inference-making from experimental designs must 
meet two other requirements. The first is referred to as control. All factors, with the exception of the 
independent variable, must be held constant and not confounded with another variable that is not part 


>Exhibit 6-5 Mill’s Method of Difference 


Descriptive Factors Outcome 
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of the study. Second, each person in the study must have an equal chance for exposure to each level of 
the independent variable. This is random assignment of subjects to groups. 

If we consider the possible relationships that can occur between two variables, we can conclude 
there are three possibilities: 


e Symmetrical 
e Reciprocal 


e Asymmetrical!® 


A symmetrical relationship is one in which two variables fluctuate together, but we assume the 
changes in neither variable are due to changes in the other. Symmetrical conditions are most often 
found when two variables are alternate indicators of another cause or independent variable. We might 
conclude that a correlation between low work attendance and active participation in a company camp- 
ing club is the result of (dependent on) another factor, such as a lifestyle preference. 

A reciprocal relationship exists when two variables mutually influence or reinforce each other. 
This could occur if the reading of an advertisement leads to the use of a brand of product. The usage, in 
turn, sensitizes the person to notice and read more of the advertising of that particular brand. 

Most research analysts look for asymmetrical relationships. With these we postulate that changes 
in one variable (the independent variable, or IV) are responsible for changes in another variable (the 
dependent variable, or DV). The identification of the [V and DV is often obvious, but sometimes the 
choice is not clear. In these latter cases we evaluate independence and dependence on the basis of: 


1. The degree to which each variable may be altered. The relatively unalterable variable is the 
independent variable (IV) (e.g., age, social status, present manufacturing technology). 

2. The time order between the variables. The independent variable (IV) precedes the dependent 
variable (DV). 


Exhibit 6-6 describes the four types of asymmetrical relationships: stimulus-response, 
property-disposition, disposition-behavior, and property-behavior. Experiments usually involve 


>Exhibit 6-6 Four Types of Asymmetrical Causal Relationships 


Relationship Type Nature of Relationship Examples 


Stimulus-Response An event or change results in a 


response from some object. 


¢ Achange in work rules leads to a higher level of worker 
output. 


¢« Achange in government economic policy restricts corpo- 
rate financial decisions. 


A price increase results in fewer unit sales. 


Disposition-Behavior A disposition causes a specific 


behavior. 


Opinions about a brand and its purchase. 


¢ Job satisfaction and work output. 


Moral values and tax cheating. 


Definitions: A stimulus is an event or force (e.g., drop in temperature, crash of stock market, product recall, or explosion in factory). A 
response is a decision or reaction. A property is an enduring characteristic of a subject that does not depend on circumstances for its 
activation (e.g., age, gender, family status, religious affiliation, ethnic group, or physical condition). A disposition is a tendency to respond in 
a certain way under certain circumstances (e.g., attitudes, opinions, habits, values, and drives). A behavior is an action (e.g., consumption 
practices, work performance, interpersonal acts, and other kinds of performance). 
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stimulus-response relationships. Property-disposition relationships are often studied in business and 
social science research. 
Unfortunately, most research studies cannot be carried out experimentally by manipulating 
variables. Yet we still are interested in the question of causation. Instead of manipulating and/ 
or controlling exposure to an experimental variable, we study subjects who have been exposed 
to the independent factor and those who have not and compare the results. This is known in re- 
search as ex post facto design. More will be said about this and other aspects of experimentation 
in Chapter 9. 
Causal inferences are going to be made. Although they may be neither permanent nor universal, 
these inferences allow us to build knowledge of presumed causes over time. Such empirical conclu- 


sions provide us with successive approximations to the truth. 


i 


1 


If the direction of a research project is not clear, it is often 
wise to follow a two-step research procedure. The first 
stage is exploratory, aimed at formulating hypotheses and 
developing the specific research design. The general re- 
search process contains three major stages: (1) exploration 
of the situation, (2) collection of data, and (8) analysis and 
interpretation of results. 


A research design is the strategy for a study and the plan 
by which the strategy is to be carried out. It specifies the 
methods and procedures for the collection, measure- 
ment, and analysis of data. Unfortunately, there is no 
simple classification of research designs that covers the 
variations found in practice. Some major descriptors of 
designs are 


° Exploratory versus formalized. 

° Monitoring versus communication study. 
° Experimental versus ex post facto. 

° Descriptive versus causal. 

° Cross-sectional versus longitudinal. 

° Case versus statistical. 

° Field versus laboratory versus simulation. 


° Subjects perceive no deviations, some deviations, or 
researcher-induced deviations. 


Exploratory research is appropriate for the total study in 
topic areas where the developed data are limited. In most 
other studies, exploration is the first stage of a project 
and is used to orient the researcher and the study. The 
objective of exploration is the development of hypoth- 
eses, not testing. 

Formalized studies, including descriptive and causal, 
are those with substantial structure, specific hypotheses 
to be tested, or research questions to be answered. 
Descriptive studies are those used to describe phe- 
nomena associated with a subject population or to 


estimate proportions of the population that have certain 
characteristics. 

Causal studies seek to discover the effect that a 
variable(s) has on another (or others) or why certain out- 
comes are obtained. The concept of causality is grounded 
in the logic of hypothesis testing, which, in turn, produces 
inductive conclusions. Such conclusions are probabilistic 
and thus can never be demonstrated with certainty. Cur- 
rent ideas about causality as complex processes improve 
our understanding over Mill’s canons, though we can never 
know all the relevant information necessary to prove causal 
linkages beyond a doubt. 


4 The relationships that occur between two variables may 


be symmetrical, reciprocal, or asymmetrical. Of greatest 
interest to the research analyst are asymmetrical relation- 
ships, which may be classified as any of the following 
types: 


° Stimulus-response 
e Property-disposition 
° Disposition-behavior 
e Property-behavior 


We test causal hypotheses by seeking to do three 
things. We (1) measure the covariation among variables, 
(2) determine the time order relationships among variables, 
and (3) ensure that other factors do not confound the 
explanatory relationships. 

The problems of achieving these aims differ some- 
what in experimental and ex post facto studies. Where 
possible, we try to achieve the ideal of the experimental 
design with random assignment of subjects, matching 
of subject characteristics, and manipulation and control 
of variables. Using these methods and techniques, we 
measure relationships as accurately and objectively as 
possible. 
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er 


asymmetrical relationship 138 experiment 127 qualitative techniques 129 


case study 128 exploratory study (exploration) 126 random assignment 138 


causal-explanatory study 127 field conditions 128 reciprocal relationship 138 


causal-predictive study 127 focus group 133 reporting study 127 


causation 136 formal study 126 research design 125 


communication study 127 intranet 133 secondary data 130 


control 137 laboratory conditions 128 simulation 128 


critical path method (CPM) 125 longitudinal study 128 statistical study 128 


cross-sectional study 128 monitoring 127 symmetrical relationship 1338 


descriptive study 127 participants’ perceptual two-stage design 133 


ex post facto design 127 awebonises 129 


experience survey 132 pomary data: 130 
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Making Research Decisions 
6 You have been asked to determine how hospitals prepare 
and train volunteers. Since you know relatively little about this 
subject, how will you find out? Be as specific as possible. 


Terms in Review 
1 Distinguish between the following: 
a Exploratory and formal studies. 


b Experimental and ex post facto research designs. 
7 You are the administrative assistant for a division chief in a 


large holding company that owns several hotels and theme 


c Descriptive and causal studies. 


Establishing causality is difficult, whether conclusions have 
been derived inductively or deductively. 


a Explain and elaborate on the implications of this 
statement. 


b Why is ascribing causality more difficult when conclu- 
sions have been reached through induction? 


ce Correlation does not imply causation. Illustrate this point 
with examples from business. 


Using yourself as the subject, give an example of each of 
the following asymmetrical relationships: 


a Stimulus-response 
b Property-disposition 
c_ Disposition-behavior 
d_ Property-behavior 


Why not use more control variables rather than depend 
on randomization as the means of controlling extraneous 
variables? 


Researchers seek causal relationships by either experi- 
mental or ex post facto research designs. 


a In what ways are these two approaches similar? 


b In what ways are they different? 


parks. You and the division chief have just come from the 

CEO's office, where you were informed that the guest 

complaints related to housekeeping and employee attitude 

are increasing. Your on-site managers have mentioned 

some tension among the workers but have not considered 

it unusual. The CEO and your division chief instruct you 

to investigate. Suggest at least three different types of re- 

search that might be appropriate in this situation. 

Propose one or more hypotheses for each of the following 

variable pairs, specifying which is the IV and which is the 

DV. Then develop the basic hypothesis to include at least 

one moderating variable or intervening variable. 

a_ The Index of Consumer Confidence and the business 
cycle. 

b Level of worker output and closeness of worker 
supervision. 

c Level of employee theft and the presence of video sur- 
veillance cameras. 


Bringing Research to Life 
9 Using the eight design descriptors, profile the MindWriter 


CompleteCare satisfaction study as described in this and 
preceding chapters. 


From Concept to Practice 


10 


Use the eight design descriptors in Exhibit 6-3 to profile the 
research described in the chapter Snapshots. 


From the Headlines 


11 


12 


Kraft will split off its global snack business this year, naming 
it Mondeléz International. The name came from a massive 
name suggestion program from Kraft employees, involving 
more than 1,700 names by more than 1,000 Kraft employ- 
ees over a five-month process. The name plays on the con- 
notation of “delicious world.” How would you design this 
process to ensure that viable names were produced? 


P&G recently moved its IAMS pet food division from 
Vandalia (OH) to Mason (OH). The move affected 


A Gem of a Study 


Calling Up Attendance 


Campbell-Ewald Pumps Awareness 
into the American Heart Association 


Covering Kids with Health Care 


Donatos: Finding the New Pizza 


Goodyear’s Aquatred 


Inquiring Minds Want to 
Know—NOW! 


Ohio Lottery: Innovative Research 
Design Drives Winning 
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240 workers, who were offered positions to join the 
2,000 workers already occupying the pet food division 
headquarters in Mason. A company statement indicated, 
“We're doing this to increase productivity, collaboration, 
and access to P&G’s resources/expertise.” P&G also told 
employees that it was beginning a separate, multi-month 
study on how to increase collaboration and efficiencies 
with the 250 employees still working in its pet food 
research and development complex located in Lewisburg 
(OH). What research might be included in the multi- 
month study to determine the future of the Lewisburg 
R&D facility and its employees? 


Oe 


Open Doors: Extending Hospitality to Travelers 
with Disabilities 


Proofpoint: Capitalizing on a Reporter’s Love of 
Statistics 


Ramada Demonstrates Its Personal Best™ 


Starbucks, Bank One, and Visa Launch Starbucks 
Card Duetto Visa 


State Farm: Dangerous Intersections 


Volkswagen’s Beetle 


“You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 
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learningobjectives 
After reading this chapter, you should understand .. . 


How qualitative methods differ from quantitative methods. 
The controversy surrounding qualitative research. 
The types of decisions that use qualitative methods. 


The variety of qualitative research methods. 


6 6Sometimes people are layered. There’s 
something totally different underneath than 
what’s on the surface . . . like pie. 9 9 


Joss Whedon, author and screenwriter 


bringingresearchtolife 


“Welcome back,” Jason comments as he stops by Sara 
Arens’s office. “Any problems?” 

“Thanks. Glad to be back. Moderating the Atlanta 
HealthPlus frozen-food groups was rather like slipping 
on comfortable shoes. It really helped to observe the 
Webcast of the San Francisco and Detroit groups just 
last week. Ill be starting the report preparation for 
HealthPlus shortly.” 

“Sara, I’m lucky to have someone I can count on 
with the extensive qualitative research experience that 
you have,” compliments Jason. 

“Thanks,” smiles Sara. “I see that the transcripts 
from the San Francisco group arrived in my absence. 
The Detroit ones should be here this afternoon, with 
Atlanta arriving by Wednesday. I’m just unearthing 
my observation notes from San Francisco now, so I can 
compare them with the transcripts. 

“We captured some critical comments that I’m 
sure will be helpful in developing the new HealthPlus 
ad campaign,” shares Sara as she extracts the desired 
transcripts from the file folders on her desk. From 
the size of the stack, she’ll be doing a fair amount 
of reading in the next few days. “HealthPlus was 
right when it surmised that consumers are skeptical 
that something healthy can taste good. But we’ve 
also learned there are some triggers we can use in the 
advertising to get them to embrace the idea of eating 
healthy.” 


“HealthPlus certainly seems well positioned given 
the growing concern over rampant obesity, especially 
among youth,” observes Jason. “Ill look forward to 
your insights.” 

“Do you know if Sam has started N6 yet?” asks 
Sara. 

“Is N6 the new software that provides the prelimi- 
nary analysis on the focus group transcript content?” 
asks Jason. At Sara’s affirmative nod, he responds, 
“Then, yes, our eager intern has the San Francisco 
transcript in process.” 

“Good to hear,” comments Sara. “He thinks this 
newest version of NUD*IST is impressive. And it does 
save us a lot of time.” 

“By the way, the Atlanta facility was equipped with 
the newest version of FocusVision’s VideoMarker, for 
marking the focus group video. That’s the first time 
Ive used the marking feature as a moderator. The 
facility promised me the marked CD by early next 
week. It’s going to save a lot of time in preparing the 
client presentation.” 

Jason smiles. Sara’s enthusiasm appears strong for 
anything digital or electronic. “Well, it looks like you 
have things well in hand, Sara. ll want you in the 
preliminary project meeting with LeapFrog at 4.” 

“Right,” comments Sara as she looks up from sorting 
note files and transcripts, “the learning-toy producer 
that wants concept testing—lI’ll be there.” 
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Is Qualitative Research? 


Managers basically do business research to understand how and why things happen. If the manager 
needs to know only what happened, or how often things happened, quantitative research methodolo- 
gies would serve the purpose. But to understand the different meanings that people place on their 
experiences often requires research techniques that delve more deeply into people’s hidden interpre- 
tations, understandings, and motivations. Qualitative research is designed to tell the researcher how 
(process) and why (meaning) things happen as they do. While qualitative research currently accounts 
for about 20% of research expenditures by businesses, that is set to change. With technology help- 
ing to rapidly expand the insights drawn from social media analysis (netnography), ethnography, 
crowdsourcing (tasking an undefined public with a problem or task), marketing research online 
communities (MROCs), and virtual groups, the use of qualitative research by business is expected 
to explode.! 

Qualitative research includes an “array of interpretive techniques which seek to describe, decode, 
translate, and otherwise come to terms with the meaning, not the frequency, of certain more or less 
naturally occurring phenomena in the social world.”” Qualitative techniques are used at both the data 
collection and data analysis stages of a research project. At the data collection stage, the array of tech- 
niques includes focus groups, individual depth interviews (IDIs), case studies, ethnography, grounded 
theory, action research, and observation. During analysis, the qualitative researcher uses content analy- 
sis of written or recorded materials drawn from personal expressions by participants, behavioral ob- 
servations, and debriefing of observers, as well as the study of artifacts and trace evidence from the 
physical environment. Observation as a methodology deserves special attention and is covered in detail 
in Chapter 8. 

Qualitative research aims to achieve an in-depth understanding of a situation, whether it explains 
why a person entering a Kroger grocery proceeds down each aisle in turn or heads for the rear of the 
store and chooses only alternate aisles thereafter or explains why some advertisements make us laugh 
and contribute to our commitment to a brand while others generate outrage and boycotts. Judith Langer, 
a noted qualitative researcher, indicates that qualitative research is ideal if you want to extract feelings, 
emotions, motivations, perceptions, consumer “language,” or self-described behavior.* Exhibit 7-1 
offers some examples of appropriate uses of qualitative research in business. 

Qualitative research draws data from a variety of sources, including the following:* 


° People (individuals or groups). 

e Organizations or institutions. 

e Texts (published, including virtual ones). 

e Settings and environments (visual/sensory and virtual material). 

e Objects, artifacts, media products (textual/visual/sensory and virtual material). 

e Events and happenings (textual/visual/sensory and virtual material). 

In this chapter we will focus on the qualitative methods that draw data from people and organiza- 


tions. The next chapter focuses on observation studies, which many authors consider an important 
contribution to qualitative data and which also contribute to the last four categories. 


> Qualitative versus Quantitative Research 


The Controversy 


Qualitative research methodologies have roots in a variety of disciplines, including anthropology, 
sociology, psychology, linguistics, communication, economics, and semiotics. Historically, qualita- 
tive methodologies have been available much longer—some as early as the 19th century—than the 
quantitative tools marketers rely on so heavily. Possibly because of their origins, qualitative methods 
don’t enjoy the unqualified endorsement of upper management. Many senior managers maintain that 
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What Does Cyberspace Offer for Performance Review Research? 


Many successful leaders understand that performance feedback 
is important. They know how meaningful it was to them in their 
personal and career growth. As a result, many successful orga- 
nizations have 360-degree formal feedback systems. Employ- 
ees have traditionally received this feedback through periodic 
surveys completed by subordinates, peers, and supervisors. 

An Accenture study, however, shows many middle manag- 
ers value informal feedback, because the traditional formal re- 
view process generates less specific feedback than desired and 
is not timely in its delivery. So with Generation Y moving into 
management roles, can human resources (HR) leverage its love 
of the Web as a way to enhance informal performance review? 

Employees comfortable with the Web are already using it 
to vent publicly about their job on sites like glassdoor.com and 
jobitorial.com. Others create Google discussion groups or a 
Web forum to discuss what they like and what they don’t like 
about their company, their industry, and even their boss. Some 
companies are using these anonymous tirades to identify and 
address workplace issues. 

Social networks have long been used in the hiring process. 
According to Kenexa, a human resource company that studies 
human behavior and team dynamics in the workplace, and of- 
fers software, business processes and consulting, “more than 
30 percent of the employees being hired in private organiza- 
tions come through employee referrals—the highest from any 
one particular source.” Is it a logical extension, then, to see if 


networking can be leveraged for performance review? Enterprise 
Rent-a-Car encourages its employees to add a widget about 
company jobs to their personal Facebook pages, and employ- 
ees get paid if the widgets result in a hire. 

Rypple is a company stepping into the social-networking-for- 
performance-reviews space, taking a page from Facebook and 
Twitter, to make performance appraisal research more useful. 
What’s a Rypple? Think of it as a Tweet with a purpose. With 
Rypple you can ask a question like, “What can | do to make 
YOU more effective in your role?” or “What can | do to help us 
be more effective in reaching our division’s goals?” Using con- 
tacts from existing sources like Outlook, Yahoo, Hotmail, Gmail, 
or Facebook, you select prospects to receive a brief message 
requesting feedback. The message contains a link to a short on- 
line form where the contact types in his or her response (called 
a rypple) to the single question. Feedback is aggregated, quickly 
returned, and semi-anonymous (restricted to those contacts to 
whom you sent your message). Rypple indicates 50 percent of 
those asked for feedback will provide it. 

Daniel Portillo, the senior director of personnel at Mozilla, 
makers of the Firefox browser, tried it. “It’s impossible to de- 
velop if you’re not getting constructive criticism.” He discovered 
“people are direct, but not malicious.” 


www.rypple.com; www.kenexa.com; www.glassdoor.com; 
www.jobitorial.com; www.mozilla.com 


qualitative data are too subjective and susceptible to human error and bias in data collection and in- 
terpretation. They believe such research provides an unstable foundation for expensive and critical 
business decisions. The fact that results cannot be generalized from a qualitative study to a larger 
population is considered a fundamental weakness. 

Increasingly, however, managers are returning to these techniques as quantitative techniques fall 
short of providing the insights needed to make those ever-more-expensive business decisions. Manag- 
ers deal with the issue of trustworthiness of qualitative data through exacting methodology:* 


° Carefully using literature searches to build probing questions. 


¢ Thoroughly justifying the methodology or combination of methodologies chosen. 


e Executing the chosen methodology in its natural setting (field study) rather than a highly 
controlled setting (laboratory). 


¢ Choosing sample participants for relevance to the breadth of the issue rather than how well they 
represent the target population. 


e Developing and including questions that reveal the exceptions to a rule or theory. 
° Carefully structuring the data analysis. 
¢ Comparing data across multiple sources and different contexts. 


e Conducting peer-researcher debriefing on results for added clarity, additional insights, and 
reduced bias. 
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>Exhibit 7-1 Some Appropriate Uses for Qualitative Research 


Decision Arena Questions to Be Answered 


Job Analysis e Does the current assignment of tasks generate the most productivity? 


e Does the advancement through different job levels incorporate the 
necessary training to foster the strongest performance? 


Productivity Enhancement e What actions could we take to boost worker productivity without gen- 


erating worker discontent? 


Benefits Management e Should our compensation plan be more flexible and customizable? 


e How do employees perceive wellness-prevention programs as com- 
pared to corrective health programs in terms of value? 


Process Understanding e What steps are involved in cleaning a wood floor? How is our product 
perceived or involved in this process? 


Union Representation e How do various departments perceive the current effort to unionize 
our plant? Where and what are the elements of discontent? 


The Distinction 


To understand the distinctions between qualitative and quantitative methodologies, let’s define the 
latter. Quantitative research attempts precise measurement of something. In business research, 
quantitative methodologies usually measure consumer behavior, knowledge, opinions, or attitudes. 
Such methodologies answer questions related to how much, how often, how many, when, and who. 
Although the survey is not the only methodology of the quantitative researcher, it is considered a 
dominant one. 

The purpose of qualitative research is based on “researcher immersion in the phenomenon to be 
studied, gathering data which provide a detailed description of events, situations and interaction be- 
tween people and things, [thus] providing depth and detail.”® Quantitative research is often used for 
theory testing (Will a $1-off instant coupon or a $1.50 mail-in rebate generate more sales for Kellogg’s 
Special K?), requiring that the researcher maintain a distance from the research to avoid biasing the 
results. Qualitative research—sometimes labeled interpretive research because it seeks to develop 
understanding through detailed description—often builds theory but rarely tests it. 

Besides the purpose of the research, this process sets up several key distinctions between qualita- 
tive and quantitative research, elaborated in Exhibit 7-2, including level of researcher involvement; 


Exhibit 7-2 Qualitative versus Quantitative Research 


Qualitative Quantitative 


Focus of Research 
Researcher Involvement 
Research Purpose 
Sample Design 
Sample Size 


Research Design 


Participant Preparation 


Data Type and 
Preparation 


Data Analysis 


Insights and Meaning 


Research Sponsor 
Involvement 


Feedback Turnaround 


Data Security 


Understand and interpret 
High—researcher is participant or catalyst 
In-depth understanding; theory building 
Nonprobability; purposive 

Small 


May evolve or adjust during the course of the 
project 


Often uses multiple methods simultaneously or 
sequentially 


Consistency is not expected 


Involves longitudinal approach 


Pretasking is common 


Verbal or pictorial descriptions 


Reduced to verbal codes (sometimes with com- 
puter assistance) 


Human analysis following computer or human 
coding; primarily nonquantitative 


Forces researcher to see the contextual 
framework of the phenomenon being 
measured—distinction between facts and 
judgments less clear 


Always ongoing during the project 
Deeper level of understanding is the norm; de- 


termined by type and quantity of free-response 
questions 


Researcher participation in data collection allows 
insights to form and be tested during the process 


May participate by observing research in real 
time or via taped interviews 


Smaller sample sizes make data collection faster 
for shorter possible turnaround 


Insights are developed as the research 
progresses, shortening data analysis 


More absolute given use of restricted access 
facilities and smaller sample sizes 
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Describe, explain, and predict 

Limited; controlled to prevent bias 
Describe or predict; build and test theory 
Probability 

Large 


Determined before commencing the project 
Uses single method or mixed methods 
Consistency is critical 


nvolves either a cross-sectional or a longitudinal 
approach 


No preparation desired to avoid biasing the 
participant 


Verbal descriptions 


Reduced to numerical codes for computerized 
analysis 


Computerized analysis —statistical and mathemat- 
ical methods dominate 


Analysis may be ongoing during the project 


Maintains clear distinction between facts and 
judgments 


Limited by the opportunity to probe respondents 
and the quality of the original data collection 
instrument 


Insights follow data collection and data entry, with 
limited ability to reinterview participants 


Rarely has either direct or indirect contact with 
participant 


Larger sample sizes lengthen data collection; 
Internet methodologies are shortening turnaround 
but inappropriate for many studies 


Insight development follows data collection and 
entry, lengthening research process; interviewing 
software permits some tallying of responses as 
data collection progresses 


Act of research in progress is often Known by 
competitors; insights may be gleaned by competi- 
tors for some visible, field-based studies 


Source: This exhibit was developed from material extracted from Judith Langer, The Mirrored Window: Focus Groups from a Moderator’s 
Point of View (Ithaca, NY: Paramount Market Publishing, 2001); Hy Mariampolski, Qualitative Market Research: A Comprehensive Guide 
(Thousand Oaks, CA: Sage Publications, 2001); David Carson, Audrey Gilmore, Chad Perry, and Kjell Gronhaug, Qualitative Marketing 
Research (Thousand Oaks, CA: Sage Publications, 2001); and Norman K. Denzin and Ynonna S. Lincoln, editors, The SAGE Handbook 
of Qualitative Research, 4th ed. (Thousand Oaks, CA: Sage Publications, April 27, 2011). 
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sampling methodology and size; data collection processes, including participant preparation and 
researcher and research sponsor involvement; data type and preparation; data analysis and timing; 
processes for reaching insights and meaning; time frame of insight discovery; and the level of data 
security.’ 

Unlike the case with quantitative data, both the researcher and research sponsor often have more sig- 
nificant involvement in collecting and interpreting qualitative data. The researcher may serve as a par- 
ticipant or a catalyst, as a participant observer, or as a group interview moderator. The research sponsor 
may observe (in some cases via Webcast of interviews directly to the sponsor’s desktop computer), 
influence interview questions, and add interpretations and insights during the process. By contrast, with 
large quantitative studies, the researcher who interprets the data and draws conclusions from it is rarely 
the data collector and often has no contact at all with the participant. 

Since researchers are immersed in the participant’s world, any knowledge they gain can be used to 
adjust the data extracted from the next participant. In quantitative research, identical data are desired 
from all participants, so evolution of methodology is not acceptable. 

Quantitative data often consist of participant responses that are coded, categorized, and reduced to 
numbers so that these data may be manipulated for statistical analysis. One objective is the quantitative 
tally of events or opinions, called frequency of response. Qualitative data are all about texts. Detailed 
descriptions of events, situations, and interactions, either verbal or visual, constitute the data. Data 
may be contained within transcriptions of interviews or video focus groups, as well as in notes taken 
during those interactions. But by definition they generate reams of words that need to be coded and 
analyzed by humans for meaning. While computer software is increasingly used for the coding process 
in qualitative research, at the heart of the qualitative process is the researcher—and his or her experi- 
ence—framing and interpreting the data.* 

Qualitative studies with their smaller sample sizes offer an opportunity for faster turnaround of 
findings. While speed should never be the primary reason for choosing a methodology, qualitative data 
may be especially useful to support a low-risk decision that must be made quickly. 

Multimillion-dollar strategies may lose their power if the competitor reacts too quickly. Data secu- 
rity is therefore of increasing concern. Both group and individual interviewing, the mainstay techniques 
of qualitative research, can be conducted in highly secure environments. In comparison, once a quan- 
titative survey or field observation or experiment is started, it is quickly common knowledge among a 
research sponsor’s competitors. Although the data might not be known, the area of inquiry often can be 
determined. For example, in a test market—an experimental quantitative design—a research sponsor’s 
competitors can often observe and extract insights right along with the sponsor. 


> The Process of Qualitative Research 


The process of developing a qualitative project is similar to the research process introduced in Chap- 
ter 1. However, three key distinctions suggested in the previous sections do affect the research pro- 
cess: (1) the level of question development in the management-research question hierarchy prior to 
the commencing of qualitative research, (2) the preparation of the participant prior to the research 
experience, and (3) the nature and level of data that come from the debriefing of interviewers or 
observers. 

The qualitative researcher starts with an understanding of the manager’s problem, but the 
management-research question hierarchy is rarely developed prior to the design of research methodol- 
ogy. Rather, the research is guided by a broader question more similar in structure to the management 
question. Exhibit 7-3 introduces the modifications to the research process. 

Much of qualitative research involves the deliberate preparation of the participant, called preex- 
ercises or pretasking. This step is important due to the desire to extract detail and meaning from the 
participant. A variety of creative and mental exercises draw participants’ understanding of their own 
thought processes and ideas to the surface. Some of these include: 


e Placing the product or medium for in-home use (with instructions to use the product or 
medium—e.g., a magazine—trepeatedly over the preparation period before the interview). 
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>Exhibit 7-3 Qualitative Research and the Research Process 


Exploration 


Exploration 


Research Design 
Strategy 
(type, purpose, time frame, scope, environment) 


Data Collection Sample Size & 
Design Recruiting Plan 


Data Collection & Preparation 


Research Reporting 


Management 
Decision 
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>picprofile 


When Anderson Analytics wanted to help its research participants delve into their deepest thoughts on a research topic, 
they used projective techniques that employed a colorful cast of characters. Participants choose one from the cast that most 
closely represents them or another they are trying to describe. They then attributed attitudes, feelings, emotions, and percep- 
tions to the character. “The added bonus of the characters is that some of the characters elicit similar responses [across 
research projects], so we have some idea on benchmarking,” shared Anderson Analytics managing partner Tom Anderson. 
“For instance the heavier male character with the guitar is often thought of as an outsider, ‘poser’ wanting desperately to fit in, 
generally not well liked.” www.andersonanalytics.com 


° Having the participants bring visual stimuli (e.g., family photos of areas or rooms in their homes 
that they hate to clean or have trouble decorating, or a favorite item of clothing). 


e Having the participants prepare a visual collage (e.g., taking pictures over several weeks, with 
a one-time-use camera, of their children’s favorite outfits for different purposes or situations or 
cutting pictures out of magazines that reflect how they feel when using a particular product or 
brand). 


e Having the participants keep detailed diaries of behavior and perceptions (e.g., a record of their 
step-by-step experience preparing a meal using a particular product). 


° Having the participants draw a picture of an experience (e.g., what they felt like when they last 
shopped in a particular store). 


e Having the participants write a dialog of a hypothetical experience (e.g., how a conversa- 
tion between the participant and a sales associate would progress when a complaint was not 
resolved).’ 


Pretasking is rarely used in observation studies and is considered a major source of error in quantitative 
studies. 

In quantitative research, unless a researcher is collecting his or her own data, interviewers or data 
collectors are rarely involved in the data interpretation or analysis stages. Although data collectors 
contribute to the accuracy of data preparation, their input is rarely, if ever, sought in the development 
of data interpretations. In qualitative studies, due to the higher level of involvement of both the spon- 
sor and the interviewer/data collector, these parties in the process are often debriefed or interviewed, 
with their insight adding richness to the interpretation of the data. Exhibit 7-4 provides an example of 
research question formation for a qualitative project. 
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>Exhibit 7-4 Formulating the Qualitative Research Question 


ta Exploration 


Literature Search and Expert Interviews 

e What new product categories show evidence 
of long-term strength? 

e Which product categories match our expertise 
in manufacturing? in customer profile? in 
distribution? 

e In which categories do our current competitors 

lack competency? Have superior competency? 


Sales in many of our established 
product categories are flat. We need 
to increase revenues. 


2a 
Qualitative Research 


e What new product categories pose 
the best opportunities for sales? 

e To what degree are we positioned in 
the minds of our customers to take 
advantage of these opportunities? 


Convergent Interviewing Focus Groups with 
e Industry analysts e Current customers 
e Futurists Using 

e Engineers ¢ Component sort 

e Imaginary universe 


e Does product A or B have the most 
potential for success? 

e How should each of these be 
positioned in the marketplace for 
best performance? 


> Qualitative Research Methodologies 


The researcher chooses a qualitative methodology based on the project’s purpose; its schedule, includ- 
ing the speed with which insights are needed; its budget; the issue(s) or topics(s) being studied; the 
types of participants needed; and the researcher’s skill, personality, and preferences. 


Sampling 


One general sampling guideline exists for qualitative research: Keep sampling as long as your breadth 
and depth of knowledge of the issue under study are expanding; stop when you gain no new knowledge 
or insights. That said, sample sizes for qualitative research vary by technique but are generally small. 
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A study might include just two or three focus groups or a few dozen individual depth interviews. How- 
ever unusual, one AT&T study, conducted to develop its 800 Reasons ad campaign for using AT&T 
long-distance service, used thousands of structured interviews in dozens of cities over several weeks. 
These interviews provided numerous reasons why businesses used the AT&T 800 service, and each of 
these “reasons” became the focus of a television and/or magazine ad in the multi-ad campaign.’ 

Qualitative research involves nonprobability sampling—where little attempt is made to generate a 
representative sample. Several types of nonprobability sampling are common: 


° Purposive sampling. Researchers choose participants arbitrarily for their unique characteristics 
or their experiences, attitudes, or perceptions; as conceptual or theoretical categories of partici- 
pants develop during the interviewing process, researchers seek new participants to challenge 
emerging patterns. 


¢ Snowball sampling. Participants refer researchers to others who have characteristics, experi- 
ences, or attitudes similar to or different from their own. 


° Convenience sampling. Researchers select any readily available individuals as participants. 


Interviews 


The interview is the primary data collection technique for gathering data in qualitative methodologies. 
Interviews vary based on the number of people involved during the interview, the level of structure, 
the proximity of the interviewer to the participant, and the number of interviews conducted during the 
research. 

An interview can be conducted individually (individual depth interview, or IDI) or in groups. 
Exhibit 7-5 compares the individual and the group interview as a research methodology. Both have a 
distinct place in qualitative research. 

Interviewing requires a trained interviewer (often called a moderator for group interviews) or the 
skills gained from experience. These skills include making respondents comfortable, probing for detail 
without making the respondent feel harassed, remaining neutral while encouraging the participant to 
talk openly, listening carefully, following a participant’s train of thought, and extracting insights from 
hours of detailed descriptive dialogue. Skilled interviewers learn to use their personal similarities with 


>Exhibit 7-5 A Comparison of Individual Depth Interviews and Group Interviews 


Individual Interview Group Interview 


Research Objective 


© Explore life of individual in depth * Orient the researcher to a field of inquiry and the language of the 


© Create case histories through repeated interviews over time 


° Test a survey 


field 
° Explore a range of attitudes, opinions, and behaviors 
© Observe a process of consensus and disagreement 


e Add contextual detail to quantitative findings 


Participants 

¢ Time-pressed participants or those difficult to recruit (e.g., ° Participants whose backgrounds are similar or not so dissimilar 
elite or high-status participants) as to generate conflict or discomfort 

¢ Participants with sufficient language skills (e.g., those older ° Participants who can articulate their ideas 


¢ Participants whose distinctions would inhibit participation 


° Participants who offer a range of positions on issues 


>chapter 7 Qualitative Research 


or differences from their interviewee to mine for information; similarities are used to convey sympathy 
and understanding, while differences are used to demonstrate eagerness to understand and empathize. 

In quantitative research we are more interested in the data collector’s following a prescribed proce- 
dure, whereas in qualitative research the individual conducting the interview needs a fuller understand- 
ing of the dilemma and how the insights will be used. So a skilled interviewer must be a “quick-study,” 
someone who can grasp an understanding of an issue without necessarily having prior experience with 
the product or service or being a technical expert. 

The researcher chooses either an unstructured interview (no specific questions or order of topics to 
be discussed, with each interview customized to each participant; generally starts with a participant nar- 
rative) or a semistructured interview (generally starts with a few specific questions and then follows the 
individual’s tangents of thought with interviewer probes) or a structured interview (often uses a detailed 
interview guide similar to a questionnaire to guide the question order and the specific way the questions 
are asked, but the questions generally remain open-ended). Structured interviews permit more direct com- 
parability of responses; question variability has been eliminated and thus answer variability is assumed to 
be real. Also, in the structured interview, the interviewer’s neutrality has been maintained. 

Most qualitative research relies on the unstructured or semistructured interview. The unstructured 
and semistructured interviews used in qualitative research are distinct from the structured interview in 
several ways. They: 


¢ Rely on developing a dialog between interviewer and participant. 
e Require more interviewer creativity. 
¢ Use the skill of the interviewer to extract more and a greater variety of data. 


¢ Use interviewer experience and skill to achieve greater clarity and elaboration of answers. 


Many interviews are conducted face-to-face, with the obvious benefit of being able to observe and 
record nonverbal as well as verbal behavior. An interview, however, can be conducted by phone or 
online. Phone and online interviews offer the opportunity to conduct more interviews within the same 
time frame and draw participants from a wider geographic area. These approaches also save the travel 
expenses of moving trained interviewers to participants, as well as the travel fees associated with 
bringing participants to a neutral site. Using interviewers who are fresher and more comfortable in con- 
ducting an interview—often from their home or office—should increase the quality of the interview. 
Also, depending on the group from which participants are drawn, there may be insufficient numbers to 
conduct group interviews in any one market, forcing the use of phone or online techniques. 


Interviewer Responsibilities 


The interviewer needs to be able to extract information from a willing participant who often is not 
consciously aware that he or she possesses the information desired. The actual interviewer is usually 
responsible for generating the interview or discussion guide, the list of topics to be discussed (unstruc- 
tured interview) or the questions to be asked (semistructured) and in what order (structured). In build- 
ing this guide, many interviewers employ a hierarchical questioning structure, depicted in Exhibit 7-6. 
Broader questions start the interview, designed to put participants at ease and give them a sense that 
they have a lot to contribute, followed by increasingly more specific questions to draw out detail. 

The interviewer is often responsible for generating the screening questions used to recruit partici- 
pants for the qualitative research. This preinterview uses a device similar to a questionnaire, called 
a recruitment screener. Exhibit 7-7 provides the various elements necessary for a comprehensive 
recruitment screener. Each question is designed to reassure the researcher that the person who has the 
necessary information and experiences, as well as the social and language skills to relate the desired in- 
formation, is invited to participate. Data gathered during the recruitment process are incorporated into 
the data analysis phase of the research, as recruitment data provide additional context for participants’ 
expressions. 

In general, then, the interviewer is a consultant with wide-ranging responsibilities:'! 


e Recommends the topics and questions. 


¢ Controls the interview, but also plans—and may manage—the locations and facilities for the study. 
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>Exhibit 7-6 The Interview Question Hierarchy 


Broad Issue 
What do participants consider entertaining? 


Narrow to one topic: 
What type of movie do participants consider most entertaining? 


Narrow the topic: 
What do participants find most entertaining about action movies? 


Narrow to the client’s specific interest: 


Within action movies, how do the computer-enhanced or computer-generated 
action sequences contribute to the movie’s entertainment experience? 


Source: This graphic was adapted from one developed by Judith Langer and published in The Mirrored Window: Focus Group from a Moderator’s Point 
of View (Ithaca, NY: Paramount Market Publishing, 2001), www.paramountbooks.com. 


>Exhibit 7-7 What Is Included in a Recruitment Screener? 


For best effect, qualitative research takes creative, articulate, expressive individuals. Finding appropriate participants is 
the task of the researcher. Here are some common elements addressed at this phase of the research. 


Type of Information Description 


Heading Include project name, date of interviews, identity of screener. 


Identity Information Include name of prospect, address, phone, e-mail. 
Security Questions Reveal possible participant overparticipation or conflicts of interest; similar informa- 


tion on spouse or immediate family members. 


Product/Brand Usage/Purchase Questions Establish frequency of use, purchase, loyalty, etc. 


Attitudinal and Knowledge Questions Look for breadth in perceptions, attitudes, opinions, knowledge. 


Offer/Termination Invite participation, discuss compensation and pretasking, set up interview, or indi- 
cate that the person is not right for the current study but may be right for future 
studies. 
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¢ Proposes the criteria for drawing the sample participants. 


e Writes the recruitment screener and may recruit participants. 


¢ Develops the various pretasking exercises. 


¢ Prepares any research tools (e.g., picture sorts or written exercises) to be used during the 


interview. 


e Supervises the transcription process. 


¢ Helps analyze the data and draw insights. 


e Writes or directs the writing of the client report, including extracting video clips for the oral report. 


Projective Techniques 


Because researchers are often looking for hidden or suppressed meanings, projective techniques can 
be used within the interview structures. Some of these techniques include:'” 


e Word or picture association 


e Sentence completion 


¢ Cartoons or empty balloons 


e Thematic Apperception Test 


¢ Component sorts 


e Sensory sorts 


¢ Laddering or benefit chain 


¢ Imagination exercises 


e Imaginary universe 


¢ Visitor from another planet 


¢ Personification 


e Authority figure 


e Ambiguities and paradoxes 


Participants are asked to match images, experiences, emotions, 
products and services, even people and places, to whatever is 
being studied. “Tell me what you think of when you think of 
Kellogg’s Special K cereal.” 


Participants are asked to complete a sentence. “Complete this 
sentence: People who buy over the Internet...” 


Participants are asked to write the dialog for a cartoonlike pic- 
ture. “What will the customer comment when she sees the sales- 
person approaching her in the new-car showroom.” 


Participants are confronted with a picture (usually a photograph 
or drawing) and asked to describe how the person in the picture 
feels and thinks. 


Participants are presented with flash cards containing compo- 
nent features and asked to create new combinations. 


Participants are presented with scents, textures, and sounds, 
usually verbalized on cards, and asked to arrange them by one 
or more criteria. 


Participants are asked to link functional features to their physi- 
cal and psychological benefits, both real and ideal. 


Participants are asked to relate the properties of one thing/per- 
son/brand to another. “Jf Crest toothpaste were a college, what 
type of college would it be?” 


Participants are asked to assume that the brand and its users 
populate an entire universe; they then describe the features of 
this new world. 


Participants are asked to assume that they are aliens and are con- 
fronting the product for the first time; they then describe their 
reactions, questions, and attitudes about purchase or retrial. 


Participants are asked to imagine inanimate objects with the traits, 
characteristics and features, and personalities of humans. “Jf 
brand X were a person, what type of person would brand X be?” 
Participants are asked to imagine that the brand or product is 
an authority figure and to describe the attributes of the figure. 
Participants are asked to imagine a brand as something else 


(e.g., a Tide dog food or Marlboro cereal), describing its at- 
tributes and position. 
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° Semantic mapping Participants are presented with a four-quadrant map where 
different variables anchor the two axes; they then spatially or- 
ganize health insurance options, product components, or orga- 
nizations within the four quadrants. 


e Brand mapping Participants are presented with different brands and asked to 
talk about their perceptions, usually in relation to several crite- 
ria. They may also be asked to spatially place each brand on one 
or more semantic maps. 


e Metaphor elicitation technique Participants are pretasked to collect images that reveal how they 
feel about a research topic; during an IDI, participants discuss 
each image and create a collage of their images, with emotions, 
thoughts, or perceptions noted near each image. 


Paper-based exercises often draw out less verbal members of a group. Projective techniques can 
dissipate tension caused by sensitive topics or can be useful when a change of focus in the interview 
is imminent. A well-trained interviewer is required if the research demands that one or more of these 
techniques be included within an individual depth interview or group interview. These techniques are 
also time-consuming to apply, lengthening the time frame of the individual or group interview. They 
also lengthen the data analysis time. 


Individual Depth Interviews 


An individual depth interview (IDD) is an interaction between an individual interviewer and a single 
participant. Individual depth interviews generally take between 20 minutes (telephone interviews) and 
2 hours (prescheduled, face-to-face interviews) to complete, depending on the issues or topics of inter- 
est and the contact method used. Some techniques such as life histories may take as long as five hours. 
Participants are usually paid to share their insights and ideas; $1 per minute is the budgeting rule of 
thumb for general consumers, but much higher rates are demanded by participants representing highly 
skilled professionals." 

Interviewees are often provided with advance materials via mail, fax, or the Internet. Recently, ad- 
vances in technology have encouraged the use of detailed visual and auditory aids during interviews, 
creating the methodology known as computer-assisted personal interviews (CAPIs). CAPIs often 
use a structured or semistructured individual depth interview. 

Several unstructured individual depth interviews are common in business research, including oral 
histories, cultural interviews, life histories, critical incident technique, and sequential (or chronologic) 
interviewing. Exhibit 7-8 describes these techniques and provides examples. 


Managing the Individual Depth Interview 


Participants for individual depth interviews are usually chosen not because their opinions are rep- 
resentative of the dominant opinion but because their experiences and attitudes will reflect the full 
scope of the issue under study. Participants for individual depth interviews also need to be verbally 
articulate, in order to provide the interviewer with the richness of desired detail. Primary Insights Inc. 
developed its CUE methodology to help marketers understand the performance cues that consum- 
ers use to judge a product. It uses purposive sampling to recruit individuals “with a specific interest 
in and aptitude for analytical thinking and discovering how things work.” CUE combines in-home 
product use with a diary preexercise, followed by individual depth interviews that extract what the 
participant saw, felt, heard, smelled, and sensed when interacting with the product. What evolves 
is a hierarchy of sensory cues that clients may use when modifying products to improve customer 
satisfaction."* 
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IBM’s High-Touch Strategy Is Research-Driven 


When Samuel Palmisano became chairman and chief executive 
of IBM, industry analysts didn’t expect this 30-year IBM loyal- 
ist to rock the boat. But he started a research program that re- 
sulted in a very aggressive reorganization. The IBM strategy isn’t 
quite so novel—create a strong connection between IBM and 
its customer—as is the extensive research behind it. Palmisano 
wanted his employees to talk to customers about their most trou- 
bling business problems. Rather than send the sales force to do 
that job, IBM formed teams: the sales executive in charge of the 
corporate account, a representative from the services division, 
a person from the software unit, and someone from the IBM re- 
search labs. These teams became known as “four in a box.” But 


figure out how IBM might help customers solve their pesky prob- 
lems. And what resulted from all those customer interviews? A 
reorganization of the $89 billion company into 12 industry groups 
(e.g., banking, insurance, automobiles, utilities, consumer pack- 
aged goods, telecommunications, life sciences, etc.) rather than 
its previous three divisions (software, services, and research). 
The shift is under way to make IBM an executive-level consulting 
firm rather than a technology services company. The reorganiza- 
tion has IBM’s labs, which used to focus on making machines 
calculate faster and more efficiently, refocusing on modeling pat- 
terns of human behavior to help solve business problems. 


www.ibm.com 


what each team was asked to do was think outside the box: 


Individual depth interviews are usually recorded (audio and/or video) and transcribed to provide 
the researcher with the rich detail that the methodology is used for. Interviewers are also them- 
selves debriefed to get their personal reactions to participant attitudes, insights, and the quality 
of the interview. Individual depth interviews use extensive amounts of interviewer time, in both 
conducting interviews and evaluating them, as well as facility time when premises are occupied 
for interviews. And while some respondents feel more comfortable discussing sensitive topics or 
sharing their own observations, behaviors, and attitudes with a single person, others are more forth- 
coming in group situations. 


Group Interviews 


A group interview is a data collection method using a single interviewer with more than one research 
participant. Group interviews can be described by the group’s size or its composition. 

Group interviews vary widely in size: dyads (two people), triads (three people), minigroups (two to 
six people), small groups (focus groups—6 to 10 people—unarguably the most well known of group 
interview techniques), or supergroups (up to 20 people). The smaller groups are usually used when 
the overall population from which the participants are drawn is small, when the topic or concept list 
is extensive or technical, or when the research calls for greater intimacy. Dyads also are used when 
the special nature of a friendship or other relationship (e.g., spouses, superior—-subordinate, siblings) 
is needed to stimulate frank discussion on a sensitive topic. Dyads and triads are also used frequently 
with young children who have lower levels of articulation or more limited attention spans and are thus 
more difficult to control in large groups. A supergroup is used when a wide range of ideas is needed in 
a short period of time and when the researcher is willing to sacrifice a significant amount of participant 
interaction for speed. 

In terms of composition, groups can be heterogeneous (consisting of different individuals; variety 
of opinions, backgrounds, actions) or homogeneous (consisting of similar individuals; commonal- 
ity of opinions, backgrounds, actions). Groups also can comprise experts (individuals exceptionally 
knowledgeable about the issues to be discussed) or nonexperts (those who have at least some desired 
information but at an unknown level). 

Driven by the belief that the data extracted will be richer because of the interaction, group inter- 
views are one of the few research techniques in which the participants are encouraged to interact. 
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Exhibit 7-8 Types of Research Using IDIs 


Types 


Oral History (narrative) 


Cultural Interviews 


Life Histories 


Critical Incident Technique 


Convergent Interviewing 
(convergent and 
divergent interviewing) 


Sequential Interviewing 


(chronologic interviewing) 


Ethnography 


Grounded Theory 


How Research Is Conducted 


Ask participants to relate their personal ex- 
periences and feelings related to historical 
events or past behavior. 


Ask a participant to relate his or her experi- 
ences with a culture or subculture, including 
the knowledge passed on by prior genera- 
tions and the knowledge participants have 
or plan to pass on to future generations. 


Extract from a single participant memories and 
experiences from childhood to the present 
day regarding a product or service category, 
brand, or firm. Participants are encouraged 
to share how the significant people in their 
lives talked about or were involved with the 
organization, how their attitudes or prefer- 
ences have changed over their lives with 
respect to the organization, and how their 
perceptions and preferences have been 
altered by their various life experiences. 


The participant describes: 
¢ What led up to the incident. 


¢ Exactly what he or she did or did not do that 
was especially effective or ineffective. 


e The outcome or result of this action and 
why this action was effective or what more 
effective action might have been expected. 


Experts serve as participants in a sequential 
series of IDIs; researcher refines the ques- 
tions with each interview in order to con- 
verge on the central issues or themes ina 
topic area. 


Approach the participant with questions 
formed around an anticipated series of 
activities that did or might have happened, 
in order to have the participant recall the 
detail of his or her own experience. 


Interviewer and participant collaborate in a 
field-setting participant observation and 
unstructured interview. 


Using a structured interview, each subsequent 
interview is adjusted based on the findings 
and interpretations from each previous 
interview, with the purpose to develop general 
concepts or theories with which to analyze 
the data. 


How Research Is Used 


To develop products, for example, books. 
[September 11, 2001: Stories from 55 Broad Street 
by Eddie T. Deerfield and Thomas T. Noland 
Jr. (editors); An Album of Memories: Personal Histo- 
ries from the Greatest Generation by Tom Brokaw.] 


To determine product positioning or advertising 
creation. 

(E.g., how people use baking soda leads to 
positioning the product as not just a baking 
ingredient but also a deodorizer, toothpaste 
substitute, etc.) 


To determine positioning for company prior to 
an |.D. or name change. 

(E.g., Frosted Flakes and Tony the Tiger—ad 
spots where adults feel they must appear in 
disguise because they eat a “child’s cereal.”) 


To evaluate manufacturing processes, personal 
sales and telemarketing sales programs, 
compensation or incentive programs, or 
other management-related incidents. 


To develop appropriate questions for all types 
of research (in exploratory research). 


To determine store design, advertising devel- 
opment, and product design; it is used to 
extract details related to shopping behavior, 
advertising consumption behavior, and 
product use behavior. 


To determine product redesign, advertising 
development, positioning, distribution selec- 
tion; to discover reactions and attitudes of 
striking employees. 


To determine product design or redesign and 
advertising and promotion development. 


Source: This exhibit was developed from Hy Mariampolski, Qualitative Market Research: A Comprehensive Guide (Thousand Oaks, CA: Sage 
Publications, 2001), p. 53; David Carson, Audrey Gilmore, Chad Perry, and Kjell Gronhaug, Qualitative Marketing Research (Thousand Oaks, CA: 
Sage Publications, 2001), pp. 84-89 and 152-157; Anselm Strauss and Julia Corbin, Basics of Qualitative Research: Techniques and Procedure for 
Producing Grounded Theory (Thousand Oaks, CA: Sage Publications, 1998); and Norman K. Denzin and Ynonna S. Lincoln, editors, The SAGE 
Handbook of Qualitative Research, 4th ed. (Thousand Oaks, CA: Sage Publications, April 27, 2011). 
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However, given time constraints, group interviews permit spending only limited time extracting detail 
from each participant.'> This problem is magnified when a group interview is structured to cover nu- 
merous questions or topics. 

Another drawback of the group interview is the increased difficulty recruiting, arranging, and 
coordinating group discussions. But this aggravation—which can be subcontracted to a specialist 
research supplier—is deemed a small price to pay for the insights that often are revealed by group 
interaction. 

Interviewers are tested by the challenge of managing the group’s conversation while avoiding 
interjecting themselves into the group’s process. It is also the moderator’s job to control the extrovert 
or dominant personality and ensure meaningful contributions from all others, including the most intro- 
verted or private thinkers. When control is not maintained, some members’ opinions may be suppressed 
and valuable insights lost. Sometimes an individual will be more honest with a neutral interviewer than 
with a group of peers. One example is a group of small-business owners being unwilling to divulge 
competitive strengths and weaknesses. A skilled researcher can anticipate which topics are more likely 
to obtain good results with an individual or a group interview. 

A group interview’s structure and process include moderator interaction with the group and probing 
of the group to clarify responses. As a result, the moderator may create bias in the results by sending 
verbal and nonverbal signals that some responses are more favorable than others. The moderator might 
also direct discussion down paths that are least likely to help the client. Only training, and subsequent 
experience, can overcome these potential weaknesses of group interviews. 

The skilled researcher helps the sponsor determine an appropriate number of group interviews to 
conduct. The number of groups is determined by 


e The scope of the issue(s) being studied: The broader the issue(s), the more groups needed. 


¢ The number of distinct market segments of interest: The larger the number and the greater the 
distinctions, the more groups needed. 


¢ The number of new ideas or insights desired: The larger the number, the more groups needed. 
¢ The level of detail of information: The greater the level of detail, the more groups needed. 


¢ The level of geographic or ethnic distinctions in attitudes or behavior: The greater these influ- 
ences, the more groups needed. 


¢ The homogeneity of the groups: The less homogeneity, the more groups needed. 


The general rule is: Keep conducting group interviews until no new insights are gained. Often a limited 
number of groups will suffice, or sometimes the number might grow to 8 or even 12. 

It is often preferable, depending on the topic, to run separate group interviews for different subsets 
of the target population. For example, a study on nutritional advice may begin with separate consumer 
and physician groups to determine the best ways to provide the advice. This type of homogeneous 
grouping tends to promote more intense discussion and freer interaction.'® 

Researchers caution against forming groups solely on demographic descriptors, favoring “natu- 
ral’”’ groups (like families, co-workers, church members, etc.) where the participants share an affin- 
ity base.'’ For customer groups, however, consideration should be given to such factors as gender, 
ethnicity, employment status, and education, because culture is a primary determinant of perception. 
In a recent exploratory study of discount shoppers, the attitudes about the economy and personal 
finances expressed by East Coast respondents and West Coast respondents diverged widely. The re- 
search sponsor was able to use information from group interviews to build a strategy tailored to each 
geographic area.'® 

Regardless of group composition, it is the moderator who sets the tone of the group. Homoge- 
nous groups often discover their similarities early and get along well. But with heterogeneous groups, 
the moderator must provide the ice-breaker activities that get the participants interacting with each 
other. As with individual depth interviews, the moderator is responsible for developing the recruitment 
screener and the group discussion guide. Exhibit 7-9 summarizes the facilitators and inhibitors of indi- 
vidual participation in group interviews. 

A closer look at one of the best known of group interviews, the focus group, may clarify these 
distinctions. 
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Problems within Focus Groups 


Founder and principal re- 
searcher Robert W. Kahle 
of Kahle Research Solu- 
tions Inc., in his book 
Dominators, Cynics, and 
Wallflowers, dissects typi- 
cal focus group partici- 
pants to illuminate ways 
to modify their problem 
behaviors. DominaToRS 
are all-knowing, quick to 
answer, and choose a 
seat location in order to 
challenge the moderator 
for control. Cynics dis- 
play negative behaviors 
and deride the ideas of 
others. Hostites have an 
agenda of their own and 
seek corrective action; 
they are often angry and 


combative. Intoxicateps are under the influence of something, and verbally. Finally, Co-moperators often engage participants 
fidgety and incoherent. PRoseLyTizers cannot accept that others before a discussion starts, ask questions of their own, and seek 
hold opposing opinions and try to persuade others to their opin- to befriend or support other participants. 

ion. BLATHERERS Offer long, off-topic answers and ignore mod- Why is each of these behaviors a problem and how would 


erator cues. Jokers find every comment source material for a you handle each of these problem participants? 


new joke, story, or comical facial expression. Followers tend to www.kahleresearch.com; www.paramountbooks.com 
repeat others’ opinions. WattrLowers withdraw both physically 


Focus Groups 


The term focus group was first coined by R. K. Merton in his 1956 book, The Focused Interview. The 
focus group is a panel of people (typically made up of 6 to 10 participants), led by a trained moderator, 
who meet for 90 minutes to two hours. The facilitator or moderator uses group dynamics principles to 
focus or guide the group in an exchange of ideas, feelings, and experiences on a specific topic. You’ ll 
find a sample focus group discussion guide in Appendix B. 

Focus groups are often unique in research due to the research sponsor’s involvement in the process. 
Most facilities permit the sponsor to observe the group and its dynamics in real time, drawing his or 
her own insights from the conversations and nonverbal signals he or she observes. Many facilities also 
allow the client to supply the moderator with new topics or questions that are generated by those ob- 
serving in real time. This option is generally not available in an individual depth interview, other group 
interviews, or survey research. 

Focus groups typically last about two hours but may run from one to three hours. Facilities usu- 
ally provide for the group to be isolated from distractions. Thus, the famous, or infamous, mirrored 
window allows those who are interested to observe the group while they avoid interfering with the 
group dynamics. Some facilities allow for product preparation and testing, as well as other creative 
exercises. 
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Exhibit 7-9 Factors Influencing Participant Contributions in Group Interviews 


Positive/Facilitators 


Recognition/Ego Enhancement oderator’s expressed appreciation for participant contributions that contribute to 
issue understanding; participant's open agreement with other participant comments. 


Personal Contribution Participant’s desire to be, and perception that his or her contributions are, helpful. 
Validation Participant’s need to have his or her feelings, attitudes, or ideas validated. 
Catharsis/Load-sharing Participant’s need to share something negative or bothersome with others. 

Personal Growth Participant’s desire to increase knowledge or understanding through new perspectives; 


participant’s desire for new experiences. 


Socialization Participant’s desire to meet new people and make new friends in a “safe” environment. 
Expectations Participant’s accurate understanding of the purpose of the group discussion. 

Extrinsic Rewards Participant’s value perception of fee for participation. 

Use of Abstract Terminology oderator or participant’s use of terminology or unfamiliar jargon. 

Ego Threats Participant’s challenging another participant’s knowledge of the subject. 

Political Correctness Participant’s withholding comments for fear that his or her contributions might be 


perceived as disrespectful of another’s knowledge or opinions. 


Ego Defense Participant’s withholding a comment for fear that it will make him or her appear 
unintelligent or that the opinion will be unpopular with the group. 

Memory Decay Participant’s failure to remember incidents or details of incidents. 

Embellishment Participant’s creative additions to memories of behaviors in order to participate fully or 


inflate status. 


Inarticulation/Rambling Accounts Participant’s inability to express ideas quickly or concisely. 

Confusion Participant’s lack of understanding of the issue under discussion. 

Reticence Participant’s need to be invited to participate (rather than actively volunteering 
comments). 

Time Participant’s concern about other obligations. 

Dominating/Monopolizing Participant’s attempting to take leadership or the spotlight, thus blocking contributions 
of others. 


Fewer and lengthier focus groups are becoming common. As sessions become longer, activities are 
needed to bring out deeper feelings, knowledge, and motivations. Besides the creativity sessions that 
employ projective techniques or involve the participants in writing or drawing sessions, or creating 
visual compilations, other common activities within focus groups include:!° 


Free association. “What words or phrases come to mind when you think of X?” 


Picture sort. Participants sort brand labels or carefully selected images related to brand personal- 
ity on participant-selected criteria. 


Photo sort. Photographs of people are given to the group members, who are then asked: “Which 
of these people would... ?” or “Which of these people would not... ?” 


Role play. Two or more members of the group are asked to respond to questions from the van- 
tage point of their personal or assigned role. 
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Where do great new product ideas come from? If you’re Hallmark, they come from a very consumer-focused research effort. 
Monica Alderson, product management director for consumer solutions shares that Hallmark is always looking for new ways 
to fulfill its brand promise: helping people remain emotionally connected. Qualitative and quantitative research is designed to 
reveal connection dilemmas. One such dilemma was grandparents indicating “help me be there even when | can’t be there.” As 
part of its new consumer-insights organization structure, each month 50 Hallmark senior managers listen to research insights 
and product proposals that are based on these insights. This was the starting point for recordable storybooks, a collection of 
books not only read for grandchildren, but books read by parents for children, and even books read by children to their military 
parents serving abroad. www.hallmark.com 


Focus groups are often used as an exploratory technique but may be a primary methodology. In two 
such cases, a small college used focus groups to develop a plan to attract more freshmen applications, 
and a blood center used a focus group to improve blood donations.” Focus groups are especially valu- 
able in the following scenarios:*! 


e Obtaining general background about a topic or issue. 

e Generating research questions to be explored via quantitative methodologies. 
Interpreting previously obtained quantitative results. 

° Stimulating new ideas for products and programs. 

° Highlighting areas of opportunity for specific managers to pursue. 

e Diagnosing problems that managers need to address. 

e Generating impressions and perceptions of brands and product ideas. 


e Generating a level of understanding about influences in the participant’s world. 


Groups best enable the exploration of surprise information and new ideas. Agendas can be modified 
as the research team moves on to the next focus group. Even within an existing focus group, an adept 
facilitator can build on the ideas and insights of previous groups, getting to a greater depth of under- 
standing. However, because they are qualitative devices, with limited sampling accuracy, results from 
focus groups should not be considered a replacement for quantitative analyses. 


FocusVision’s VideoMarker 
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for most researchers. “The power to convince decision 
makers is often locked in the footage of such interviews,” 
shares Peter Houlahan, president and COO of Focus- 
Vision. FocusVision, a company that provides more than 
280 facilities worldwide with services for videoconference 
focus groups and individual depth interviews (IDIs), devel- 
oped new technology for this purpose: VideoMarker. Clients 
plug in their laptop (in a viewing room or their office via 
videostreaming technology). While watching the event, 
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when they see footage they want to mark, they click on 
the “VideoMark” button above the video area on their 
PC. A pop-up textbox allows the client to enter a note. 
The note is automatically coded with a time mark and the 
name of its creator. When the event is complete, video of the 
entire project and all notes are archived for immediate access (by 
password) and recorded on a CD-ROM that is sent to the client. 
Researchers can then create video clips to share with colleagues 
via e-mail, embed in documents or PowerPoint presentations, or 
group together into highlight reels. “The capability to show actual 
footage when presenting research results is especially relevant 
when clients aren’t present to watch behind a one-way mirror or 


Technical Support 
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Help 
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offers Houlahan. One pharmaceutical company arranged to 
interview patients in more than seven hours of focus groups. 
With VideoMarker, researchers captured the highlights, which were 
then—with the patients’ consent—incorporated into IDI research 
with physicians. “These research highlights were the key motiva- 
tional tool used to persuade doctors to change how they pre- 
scribed the company’s product.” 


www.focusvision.com 


when they participate via videoconferencing or videostreaming,” 


In the opening vignette, Sara Arens was involved in conducting and analyzing focus groups for 
a frozen-food manufacturer. Sara’s partner Jason is involved with assessing the CompleteCare ser- 
vice program for MindWriter. For the latter project Jason and Sara could use focus groups involving 
employees (of the call center and service departments) to determine suggestions for improvements and 
provide an analysis of proposed improvements. MindWriter may want focus groups with CompleteCare 
customers (both dissatisfied and satisfied customers but restricted to separate groups) to reveal the 
scope of attitudes and experiences not documented within complaints. 


Other Venues for Focus Group Interviews 


Although the following venues are most frequently used with focus groups, they can be used with other 
sizes and types of group interviews. 


Telephone Focus Groups In traditional focus groups, participants meet face-to-face, usually in 
specialized facilities that enable respondents to interact in a comfortable setting while being observed 
by a sponsoring client. However, often there is a need to reach people that face-to-face groups cannot 
attract. With modern telephone conferencing facilities, telephone focus groups can be particularly 
effective in the following situations: 


¢ When it is difficult to recruit desired participants—members of elite groups and hard-to-find respon- 
dents such as experts, professionals, physician specialists, high-level executives, and store owners. 
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e When target group members are rare, “low incidence,” or widely dispersed geographically— 
directors of a medical clinic, celebrities, early adopters, and rural practitioners. 


e When issues are so sensitive that anonymity is needed but respondents must be from a wide 
geographic area—people suffering from a contagious disease, people using nonmainstream 
products, high-income individuals, competitors. 


e When you want to conduct only a couple of focus groups but want nationwide representation. 


Telephone focus groups are usually shorter than traditional groups, averaging about one hour. Par- 
ticipants could be in their own offices or homes or be brought to a central location with the necessary 
equipment. Telephone focus groups are usually less expensive than face-to-face focus groups—by up 
to 40 percent. 

In contrast to face-to-face groups, heterogeneous telephone groups can be productive. People in tra- 
ditional superior-subordinate roles can be mixed as long as they are not from the same city. A telephone 
focus group is less likely to be effective under the following conditions: 


e When participants need to handle a product. 
e When an object of discussion cannot be sent through the mail in advance. 
e When sessions will run long. 


e When the participants are groups of young children. 


Online Focus Groups An emerging technique for exploratory research is to approximate 
group dynamics using e-mail, websites, blogs, online communities, or an Internet chat room. It is 
possible to do “live” voice chats online, reducing or eliminating the cost associated with telephone 
focus groups. Posting questions to a newsgroup with an interest in the research problem can generate 
considerable discussion. However, online discussions are not confidential unless they take place on 
an intranet. Although online forum discussions are unlikely to reflect the average participants, they 
can be a good way of getting in touch with populations that have special interests (e.g., BMW club 
members, little league coaches, or “power computer users”). Online focus groups have also proved 
to be effective with teens and young adults, as well as technically employed segments of the market, 
those essentially comfortable with computer use. They are especially valuable when a computer- 
based application, such as software or a game, is the topic of group discussion. The technology per- 
mits use of visual images of materials (e.g., ads or product concepts) but retains the barrier between 
the group and the moderator. Online focus groups are a trade-off. What you gain in speed and access, 
you give up in insights extracted from group dynamics, the flexibility to use nonverbal language as a 
source of data, and the moderator’s ability to use physical presence to influence openness and depth 
of response. 


Videoconferencing Focus Groups Videoconferencing is another technology used with 
group interviews. Many researchers anticipate growth for this methodology. Like telephone focus 
groups, videoconferencing enables significant savings. By reducing the travel time for the moderator 
and the client, coordinating such groups can be accomplished in a shorter time. However, videoconfer- 
encing retains the barrier between the moderator and participants, although less so than the telephone 
focus group. Since large corporations and universities are more likely to have their own internal video- 
conferencing facilities, most videoconferencing focus groups will tend to occur within this setting, thus 
reducing the breadth of participants to those who can access these specialized facilities. The advent of 
video technology on phones, tablets, and laptops, in combination with conferencing services like Skype 
and WebEx, have extended this methodology to smaller organizations as well. 


Recording, Analyzing, and Reporting Group Interviews 


In face-to-face settings, some moderators use large sheets of paper on the wall of the group room to 
record trends; others use a personal notepad. Facility managers produce both video- and audiotapes to 
enable a full analysis of the interview. The verbal portion of the group interview is transcribed along 
with moderator debriefing sessions and added to moderator notes. These are analyzed across several 


Mystery Shopping at Office Depot 
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You can’t use research to improve your company if you are 
asking the wrong questions. That’s the lesson learned by CEO 
Kevin Peters when he was comparing declining sales to stellar 
mystery shopping reports on customer service. 

Mystery shopping is a qualitative technique that uses ethnog- 
raphy to evaluate the in-store customer experience. The shop- 
ping experience is guided by a checklist that can contain a list 
of questions to answer or shopping scenarios to enact, or both. 
Once the mystery shopper has completed the task, he or she 
writes up a report. The technique has been used for decades by 
bricks-and-mortar retailers. 

But it wasn’t working as it should have for Office Depot, not 
because the technique is flawed but because the questions/ 


reenacting the same scenario with a twist—he followed cus- 
tomers out to the parking lot and asked them why they hadn’t 
made a purchase if their hands were empty. “Our mystery- 
shopping scores were correct. You know what was flawed? 
Our scoring system,” discovered Peters. “We were asking: Are 
the floors clean? Are the shelves full of inventory? Are the store 
windows clean? Have the bathrooms been cleaned recently? 
It turns out that customers don’t really care about any of that. 
Those factors don’t drive purchases, and that’s why our sales 
were declining.” 

Peters discoveries lead to numerous changes to Office Depot — 
store size, layout, service mix, and employee training. And we are 
positive he also changed the tasking of the mystery shopper. 


scenarios were flawed. 
Peters set out find out what was wrong by becoming a mys- 
tery shopper himself in more than 70 stores, covering 15 states, 


www.officedepot.com 


focus group sessions using content analysis. This analytical process provides the research sponsor 
with a qualitative picture of the respondents’ concerns, ideas, attitudes, and feelings. The preliminary 
profile of the content of a group interview is often done with computer software in content analysis (for 
example, N6, mentioned in the opening vignette). Such software searches for common phrasing and 
words, context, and patterns of expression on digitized transcripts. 


> Combining Qualitative Methodologies 
Case Study” 


The case study, also referred to as the case history, is a powerful research methodology that combines 
individual and (sometimes) group interviews with record analysis and observation. Researchers extract in- 
formation from company brochures, annual reports, sales receipts, and newspaper and magazine articles, 
along with direct observation (usually done in the participant’s “natural” setting), and combine it with 
interview data from participants. The objective is to obtain multiple perspectives of a single organization, 
situation, event, or process at a point in time or over a period of time. Case study methodology—or the 
written report from such a research project, often called a case analysis or case write-up—can be used 
to understand particular processes. For example, one study might evaluate new product development 
processes for similarities, especially the use of outside consultants, ideational techniques, and computer 
simulation. Another study might examine in detail the purchaser’s response to a stimulus like a display. 
The results of the research could be used to experiment with modifications of the new product develop- 
ment process or with display selection and placement processes to generate higher-value transactions. 
The research problem is usually a how and why problem, resulting in a descriptive or explanatory study. 

Researchers select the specific organizations or situations to profile because these examples or sub- 
jects offer critical, extreme, or unusual cases. Researchers most often choose multiple subjects, rather 
than a single subject, to study because of the opportunity for cross-case analysis. In studying multiple 
subjects, a deeper understanding of the subject emerges. When multiple units are chosen, it is because 
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they offer similar results for predictable reasons (literal replication) or contrary results for predict- 
able reasons (theoretical replication). While theoretical sampling seems to be common, a minimum of 
4 cases with a maximum of 15 seems to be favored. 

In the case study, interview participants are invited to tell the story of their experience, with those 
chosen representing different levels within the same organization or different perspectives of the same 
situation or process to permit depth of perspective. The flexibility of the case study approach and the 
emphasis on understanding the context of the subject being studied allow for a richness of understand- 
ing sometimes labeled thick description. 

During analysis, a single case analysis is always performed before any cross-case analysis is con- 
ducted. The emphasis is on what differences occur, why, and with what effect. Prescriptive inferences 
about best practices are concluded after completing case studies on several organizations or situations 
and are speculative in nature. 

Students are quite familiar with studying cases as a means of learning business principles. In Search 
of Excellence, a book by Tom Peters and Robert Waterman, was developed using case study methodol- 
ogy.” Other similar studies profiled in books written on Procter & Gamble and Disney have also used 
this methodology. In the business arena, such case studies have examined changes in new product 
development, sales processes, hiring practices, and training programs. 


Action Research 


Managers conduct research in order to gain insights to make decisions in specific scenarios. Action 
research is designed to address complex, practical problems about which little is known—thus no 
known heuristics exist. So the scenario is studied; a corrective action is determined, planned, and 
implemented; the results of the action are observed and recorded; and the action is assessed as ef- 
fective or not. The process is repeated until a desired outcome is reached, but along the way much 
is learned about the processes and about the prescriptive actions being studied. Action researchers 
investigate the effects of applied solutions. Whatever theories are developed are validated through 
practical application.” 

Suppose a restaurant that had never received a customer complaint earns its first challenge by a 
disgruntled diner. If no general rule existed about how to treat unhappy patrons, the organization could 
study the situation and come up with alternative actions. It might: 


¢ Ignore the problem. (Its lack of experience would prevent it from knowing that negative word of 
mouth—negative buzz—would be the likely result.) 


¢ Do whatever is necessary to replace the unsatisfactory meal within the shortest period of time. 


e Accept the current circumstance as uncorrectable, apologize to the customer, and remedy the situ- 
ation by picking up the table’s full dining tab and offering the customer a free meal to get him or 
her back in the restaurant another day. 


In action research, one of those alternatives would be chosen and implemented, and then the results 
recorded. Was the customer happy when he or she left? Did the customer return to dine another evening 
or never return again? Over the next three months, what was the customer’s full revenue value? If the 
customer didn’t return, the next time a disgruntled customer voiced dissatisfaction, a different action 
would be chosen, implemented, and then assessed in comparison to the first option’s results. 


> Merging Qualitative and Quantitative 


Methodologies 


Triangulation is the term used to describe the combining of several qualitative methods or combining 
qualitative with quantitative methods. Because of the controversy described earlier, qualitative studies 
may be combined with quantitative ones to increase the perceived quality of the research, especially 
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when a quantitative study follows a qualitative one and provides validation for the qualitative findings. 
Four strategies for combining methodologies are common in business research: 


1. Qualitative and quantitative studies can be conducted simultaneously. 


2. A qualitative study can be ongoing while multiple waves of quantitative studies are done, mea- 
suring changes in behavior and attitudes over time. 


3. A qualitative study can precede a quantitative study, and a second qualitative study then might 
follow the quantitative study, seeking more clarification. 


4. A quantitative study can precede a qualitative study. 


An example of the first strategy would be the combination of a public opinion poll at the time focus 
groups are being held to discover ways to sway a particular public’s opinion. For the second strategy, 
we might collect life histories while multiple waves of questionnaires are measuring the response to 
differing promotional tactics. For the third, we could perform a qualitative study to identify peoples’ 
behaviors and perceptions with respect to furniture shopping processes and interior decorating; then we 
could use that information to develop a quantitative study to measure the actual frequency of behaviors 
and attitudes. And, fourth, we might survey people’s behavior and attitudes toward a brand and find we 
need some IDIs to explain findings that are unclear. 

Many researchers recognize that qualitative research compensates for the weaknesses of quantita- 
tive research and vice versa. These forward thinkers believe that the methodologies complement rather 
than rival each other. 


1 Qualitative research includes an array of interpretive tech- testing; sales analysis; sales development; package design; 


niques that seek to describe, decode, translate, and other- 
wise come to terms with the meaning, not the frequency, 
of certain more or less naturally occurring phenomena in 
the social world. Qualitative research methodologies differ 
from quantitative methodologies based on the focus of the 
research; its purpose; researcher involvement; sampling 
design; sample size; research design, including participant 
pretasking; data source, type, and preparation; methods 
of data analysis; level of insights and meaning extracted; 
research sponsor involvement; speed of the research; and 
data security. A qualitative methodology may be used alone 
to address organizational problems or in combination with 
other qualitative or quantitative methodologies. 


While qualitative research is being used increasingly be- 
cause of the methodologies’ ability to generate deeper 
understanding, it still is perceived by many senior-level ex- 
ecutives as a stepchild of quantitative data collection. This is 
primarily due to qualitative research’s use of nonprobability 
sampling, the smaller sample sizes involved, and the non- 
projectability of the results to a broader, target population. 


Qualitative research is designed to tell the researcher how 
(process) and why (meaning) things happen as they do. In 
business planning and decision making, qualitative method- 
ologies are used in market segmentation; advertising creative 
development; new product development, especially concept 


brand development and assessment, especially understand- 
ing brand value; positioning; retail design; and understanding 
various processes, including consumers’ decision-making 
processes. In data analysis, qualitative research uses content 
analysis of written or recorded materials drawn from personal 
expressions by participants, behavioral observations, and 
debriefing of observers, as well as the study of artifacts and 
trace evidence from the physical environment. 


Qualitative methodologies used in decision making evolved 
from techniques used in anthropology, sociology, psychology, 
linguistics, communication, economics, and semiotics. Com- 
mon among these strategies are the individual depth interview 
(IDI) and the group interview, as well as observation, ethnogra- 
phy, action research, and grounded theory. Within group inter- 
views, the focus group is the most widely used methodology. 

Qualitative research often uses projective techniques, 
designed to encourage the participant to reveal in detail 
deeply suppressed attitudes, opinions, feelings, and experi- 
ences. Among these techniques are word or picture asso- 
ciation, sentence completion, cartoons or empty balloons, 
the Thematic Apperception Test, imagination exercises, and 
sorting exercises. Participant preparation and the actual 
qualitative sessions themselves often include various cre- 
ativity sessions and exercises. 
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er 


action research 166 
case study (case history) 165 


computer-assisted personal interview 
(CAPI) 156 


content analysis 165 
focus groups 160 
online 164 
telephone 163 
videoconferencing 164 
group interview 157 
expert group 157 
heterogeneous group 157 
homogeneous group 157 
nonexpert group 157 


individual depth interview 
(IDI) 156 


convergent interviewing 158 
critical incident technique 158 
cultural interviews 158 
ethnography 158 

grounded theory 158 

life histories 158 

oral history (narrative) 158 


sequential interviewing 
(chronologic interviewing) 158 


>QIi ion | 


Terms in Review 


1 How does qualitative research differ from quantitative 


research? 


2 How do data from qualitative research differ from data in 


quantitative research? 


interview 152 qualitative research 144 


semistructured 153 quantitative research 146 
structured 153 recruitment screener 153 
unstructured 153 triangulation 166 
interview guide (discussion guide) 153 
moderator 152 
nonprobability sampling 152 
pretasking 148 
projective techniques 155 

ambiguities and paradoxes 155 

authority figure 155 

cartoons or empty balloons 155 

component sorts 155 

imagination exercises 155 

imaginary universe 155 

personification 155 

visitor from another planet 155 

laddering or benefit chain 155 

metaphor elicitation technique 156 

semantic mapping 156 

brand mapping 156 

sensory sorts 155 

sentence completion 155 

Thematic Apperception Test 155 


word or picture association 155 


Making Research Decisions 
5 Assume you are a manufacturer of small kitchen electrics, 
like Hamilton Beach/Proctor Silex, and you want to de- 
termine if some innovative designs with unusual shapes 
and colors developed for the European market could be 
successfully marketed in the U.S. market. What qualitative 


3 Why do senior executives feel more comfortable relying on 
quantitative data than qualitative data? How might a quali- 
tative research company lessen the senior-level executive’s 
skepticism? 

4 Distinguish between structured, semistructured, and un- 
structured interviews. 


research would you recommend, and why? 


6 NCR Corporation, Known as a world leader in ATMs, 
point-of-sale (POS) retail checkout scanners, and check- 
in kiosks at airports, announced in June 2009 that it 
would move its world headquarters from Dayton (OH) 
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to Duluth (GA), a suburb of Atlanta, after more than 125 From Concept to Practice 
years. An employer of 1,200 mostly high-salaried, profes- 8 Use Exhibit 7-7 to develop the recruitment screener for the 
sional workers in Dayton, NCR was enticed to move by research you described in your answer to question 5. 


Georgia's offer of more than $56.9 million in tax credits; 
its fast-growing, educated 25- to 34-year-old population 
cohort; international offices for 10 European state govern- 
ments; and the busiest international airport (Atlanta) in the 
world. 


9 Conduct a focus group among students in your class on 
one of the following topics: 


a Our department’s problems offering requirements 
and electives essential for meeting your graduation 


expectations. 
a What qualitative research might NCR have done to 


reach thie decision? b Entertainment sponsored by your university to bring the 


community on campus. 
b NCR will use its move to Georgia to downsize its world 


headquarters workforce. What qualitative research From the Headlines 
could help NCR determine which of its 1,200 employees 10 Pepsi launched a new global branding campaign based 
will be offered positions in Duluth? on the concept of live in the moment called “Live For 
Bringing Research to Life Now.” It did extensive research prior to the campaign’s 
7 What dilemma does HealthPlus face, and why has the launch. What research should be done to determine if the 
company turned to focus groups for insights? campaign is resonating with worldwide audiences? 


CO 


Akron Children’s Hospital Open Doors: Extending Hospitality to Travelers 
with Disabilities 


Ramada Demonstrates Its Personal Best™ 


ee Covering Kids with Health Care 
& Lexus SC 430 
Starbucks, Bank One, and Visa Launch Starbucks 
om Card Duetto Visa 
NCRCC: Teeing Up a New Strategic 
Direction 


& USTA: Come Out Swinging 


ee Ohio Lottery: Innovative Research 
Design Drives Winning 


*You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 


chapter 8 


learningobjectives 
After reading this chapter, you should understand . . . 


When observation studies are most useful. 

The distinctions between monitoring nonbehavioral and behavioral activities. 

The strengths of the observation approach in research design. 

The weaknesses of the observation approach in research design. 

The three perspectives from which the observer—participant relationship may be viewed in observation studies. 


The various designs of observation studies. 


6 6 Today, scores of built-in sensors in Formula One cars 
continuously capture a multiplicity of variables— 
including RPMS, weather, road conditions, the angles 
on atrack’s s-bends, wear on tires, to pit stop now or 
next lap—and process it into data for active simulation 


models that inform a drivers’ instantaneous decisions. 
Formula One racing today is as much about deriving 
Continuous Intelligence and decision-making as it is 
about engine speed and driving expertise. 9 9 


John Chen, CEO, 
Sybase 


bringingresearcnhtolife 


“How’s the HomeExtravaganza project coming?” asks 
Jason as he sticks his head into Sara Arens’s office. 

“T finished reviewing the proposals yesterday and 
selected MarketViews as the subcontractor to do the 
observation study,” responds Sara. “MarketViews will 
start a week after the checklist is finalized—that’s the 
next step.” 

“You’ve obviously determined how the observation 
study will interact with the larger shopper motivation 
study we’re doing. Fill me in.” 

“You indicated last week that early survey feedback is 
that customer confusion related to merchandise location 
and availability may be a contributing factor to declining 
repeat visits and sales. The observation study will identify 
specific types of shopper confusion in the store and the 
sales associates’ response to that confusion.” 

“T was in the store in Boca just this week for the first 
time,” remarks Jason. “The extensive product display 
is impressive, but a little overwhelming—and the store 
is mammoth, as well. I was certainly wishing ’'d worn 
my Nikes! 


“Even though HomeExtravaganza uses greeters 
and its advertising promises lots of helpful associates, 
the motivation study is logging complaints that the 
associates aren’t as helpful as they need to be,” 
continues Jason. 

“MarketViews recommends participant observation 
to determine just what form associate help is taking.” 

“T’ve used MarketViews before with good results.” 

“Tm meeting with MarketViews’ project director 
this afternoon to rough out the checklist. So ’'m 
taking an early lunch and plan to visit the store one 
more time,” explains Sara. ““We want to select specific 
locations for the interactions to take place and specific 
behaviors . . . like walking the customer to the aisle 
location versus giving directions to the location, 
finding the item in the aisle versus leaving the 
customer at the aisle entry. And I’ve come prepared,” 
says Sara as she comes from behind her desk wearing 
her Reebok running shoes. 

“Tl leave you to your own observation study then,” 
remarks Jason. 
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> The Uses of Observation 


Much of what we know comes from observation. We notice co-workers’ reactions to political intrigue, 
the sounds of the assembly area, the smell of perfume, the taste of office coffee, the smoothness of the 
vice president’s marble desk, and a host of other stimuli. While such observation may be a basis for 
knowledge, the collection processes are often haphazard. 

Observation qualifies as scientific inquiry when it is conducted specifically to answer a research 
question, is systematically planned and executed, uses proper controls, and provides a reliable and 
valid account of what happened. The versatility of observation makes it an indispensable primary 
source method and a supplement for other methods. Many academics have a limited view of 
observation, relegating it to a minor technique of field data collection. This ignores its potential for 
forging business decisions and denies its historic stature as a creative means of obtaining primary 
data. Exhibit 8-1 depicts the use of observation in the research process. 

In Chapter 6, we said that research designs are classified by the approach used to gather primary data: 
We can observe, or we can communicate. Exhibit 8-2 describes the conditions under which observation 
is an appropriate method for data collection. It also contrasts those conditions with ones from the com- 
munication modes discussed in Chapter 7—interviews—and Chapter 10—surveys (see Exhibit 8-2). 


>Exhibit 8-1 Observation and the Research Process 


Select 
Observation 


Approach 


Exhibit 8-2 


Research Design 


Select Type of Observation 
(see Exhibit 8-3) 


Observation 
by humans 


Observation 
subject 
Select Sample Element and unavailable Reschedule 
Schedule Observation Observation 


Observation 
subject 
available 


Verify Observation 
ee Collect & Process Data 


<—m——_ Exhibit 8-6 ————————_— 
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>Exhibit 8-2 Selecting the Data Collection Method 
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Besides collecting data visually, observation involves listening, reading, smelling, and touching. 
Behavioral scientists define observation in terms of animal or human behavior, but this too is limiting. 
As used in this text, observation includes the full range of monitoring behavioral and nonbehavioral 
activities and conditions, which, as shown in Exhibit 8-3, can be classified roughly as follows: 


Nonbehavioral Observation 


¢ Record analysis 
e Physical condition analysis 


e Physical process analysis 


Behavioral Observation: 


¢ Nonverbal analysis 
e Linguistic analysis 
¢ Extralinguistic analysis 


° Spatial analysis 


Nonbehavioral Observation 


A prevalent form of observation research is record analysis. This may involve historical or current 
records and public or private records. They may be written, printed, sound-recorded, photographed, or 
videotaped. Historical statistical data are often the only sources used for a study. Analysis of current 
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>Exhibit 8-3 Selecting an Observation Approach 
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Behavioral 


Direct versus indirect 
Open versus concealed 
Participant versus 
nonparticipant 


financial records and economic data also provides a major data source for studies. Other examples of 
this type of observation are the content analysis (described in Chapter 15) of competitive advertising 
and the analysis of personnel records. 

Physical condition analysis is typified by store audits of merchandise availability, studies of plant 
safety compliance, analysis of inventory conditions, and analysis of financial statements. Process or 
activity analysis includes time/motion studies of manufacturing processes and analysis of traffic flows 
in a distribution system, paperwork flows in an office, and financial flows in the banking system. 
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Observation as a research technique can sometimes be undervalued by researchers, but Best Buy has a different approach. 
It challenges its store associates to identify ways to take advantage of hyper-local initiatives. One such initiative took place in 
the port city of Houston, where associates noticed an influx of eastern European ship workers purchasing iPods and laptops. 
Based on the associates’ observations, Best Buy reallocated floor space in the Baytown Houston store, moving desired items 
to the front of the store and merchandising accessories like international rechargers nearby. Associates combined shopping 
observations with additional ones: that any given ship was often in port a limited time and their store was a 15-minute cab ride 
from the port. Sales increased 67 percent as a result of the change. Appreciative ship workers revealed to Best Buy personnel 
that they were stocking up for family and friends when their ship was in port because similar items were more expensive in their 
home country. www.bestbuy.com 


Behavioral Observation 


The observational study of persons can be classified into four major categories.' Nonverbal behavior 
is the most prevalent of these and includes body movement, motor expressions, and even exchanged 
glances. At the level of gross body movement, one might study how a salesperson travels a territory. 
At a fine level, one can study the body movements of a worker assembling a product or time-sample 
the activity of a department’s workforce to discover the share of time each worker spends in various 
ways. More abstractly, one can study body movement as an indicator of interest or boredom, anger or 
pleasure in a certain environment. Motor expressions such as facial movements can be observed as a 
sign of emotional states. Eyeblink rates are studied as indicators of interest in advertising messages. 
Exchanged glances are of interest in studies of interpersonal behavior. 

Linguistic behavior is a second frequently used form of behavior observation. One simple type 
familiar to most students is the tally of “ahs” or other annoying sounds or words a professor makes 
or uses during a class. More serious applications are the study of a sales presentation’s content or the 
study of what, how, and how much information is conveyed in a training situation. A third form of 
linguistic behavior involves interaction processes that occur between two people or in small groups. 
Bales has proposed one widely used system for classifying such linguistic interactions.” 

Behavior also may be analyzed on an extralinguistic level. Sometimes extralinguistic behavior 
is as important a means of communication as linguistic behavior. One author has suggested there 
are four dimensions of extralinguistic activity. They are (1) vocal, including pitch, loudness, and 
timbre; (2) temporal, including the rate of speaking, duration of utterance, and rhythm; (3) interaction, 
including the tendencies to interrupt, dominate, or inhibit; and (4) verbal stylistic, including vocabulary 
and pronunciation peculiarities, dialect, and characteristic expressions. These dimensions could add 
substantial insight to the linguistic content of the interactions between supervisors and subordinates or 
salespeople and customers. 

A fourth type of behavior study involves spatial relationships, especially how a person relates 
physically to others. One form of this study, proxemics, concerns how people organize the territory 
about them and how they maintain discrete distances between themselves and others. A study of how 
salespeople physically approach customers and a study of the effects of crowding in a workplace are 
examples of this type of observation. 

Often in a study, the researcher will be interested in two or more of these types of information and 
will require more than one observer. In these forms of behavior study, it is also important to consider 
the relationship between observers and participants. 
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> Evaluation of the Observation Method 


Observation is the only method available to gather certain types of information. The study of records, 
mechanical processes, and young children, as well as other inarticulate participants, falls into this cat- 
egory. Another value of observation is that we can collect the original data at the time they occur. We 
need not depend on reports by others. Every respondent filters information no matter how well inten- 
tioned he or she is. Forgetting occurs and there are reasons why the respondent may not want to report 
fully and fairly. Observation overcomes many of these deficiencies of questioning. 

A third strength is that we can secure information that most participants would ignore either because 
it is so common and expected or because it is not seen as relevant. For example, if you are observing 
buying activity in a store, there may be conditions important to the research study that the shopper 
does not notice or consider important, such as: What is the weather? What is the day of the week or the 
time of the day? How heavy is customer traffic? What is the level of promotional activity in competing 
stores? We can expect to learn only a few of the answers to these questions from most participants. 

The fourth advantage of observation is that it alone can capture the whole event as it occurs in its 
natural environment. Whereas the environment of an experiment may seem contrived to participants, 
and the number and types of questions limit the range of responses gathered from respondents, obser- 
vation is less restrictive than most primary collection methods. Also, the limitations on the length of 
data collection activities imposed by surveys or experiments are relaxed for observation. You may be 
interested in all the conditions surrounding a confrontation at a bargaining session between union and 
management representatives. These sessions may extend over time, and any effort to study the unfold- 
ing of the negotiation is facilitated by observation. Questioning could seldom provide the insight of 
observation for many aspects of the negotiation process. 

Finally, participants seem to accept an observational intrusion better than they respond to question- 
ing. Observation is less demanding of them and normally has a less biasing effect on their behavior 
than does questioning. In addition, it is also possible to conduct disguised and unobtrusive observation 
studies much more easily than disguised questioning. 

The observation method has some research limitations. The observer normally must be at the scene 
of the event when it takes place, yet it is often impossible to predict where and when the event will 
occur. One way to guard against missing an event is to observe for prolonged periods until it does 
occur, but this strategy brings up a second disadvantage. Observation is a slow and expensive process 
that requires either human observers or costly surveillance equipment. 

A third limitation of observation is that its most reliable results are restricted to information that can 
be learned by overt action or surface indicators. To go below the surface, the observer must make infer- 
ences. Two observers will probably agree on the nature of various surface events, but the inferences 
they draw from such data are much more variable. 

Fourth, the research environment is more likely suited to subjective assessment and recording of data 
than to controls and quantification of events. When control is exercised through active intervention by 
the researchers, their participation may threaten the validity of what is being assessed. Even when sample 
sizes are small, the observation records can be disproportionately large and difficult to analyze. 

Fifth, observation is limited as a way to learn about the past. It is similarly limited as a method by 
which to learn what is going on in the present at some distant place. It is also difficult to gather informa- 
tion on such topics as intentions, attitudes, opinions, or preferences. Nevertheless, any consideration of 
the merits of observation confirms its value when used with care and understanding. 


> The Observer—-Participant Relationship 


Interrogation presents a clear opportunity for interviewer bias. The problem is less pronounced with 
observation but is still real. The relationship between observer and participant may be viewed from 
three perspectives: 

e Whether the observation is direct or indirect. 

e Whether the observer’s presence is known or unknown to the participant. 


e What role the observer plays. 
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Magnetic resonance imaging has been used in hospitals for iden- 
tifying diseases and pre-planning surgeries, but electroencepha- 
lography (EEG)—which measures the electrical field outside the 
head via a headband—and galvanic skin response (GSR) as 
used for business research have been restricted to experiments. 
Now the positron emission tomography (PET) scan is offering 
the researcher a glimpse into the respondent’s brain—one that’s 
colorful and provides literally thousands of data points. Known 
as the Christmas-tree effect, the PET scan assigns different 
types of brain activity unique colors, so the brain literally lights up 
a researcher’s understanding. 

“Neuroimaging techniques can contribute to fundamental 
research in terms of a better conceptual understanding of latent 
constructs in behavior,” claims Monika Koller, Vienna University 
of Economics and Business Administration (Austria). But Sean 
Green (cognitive psychologist) and Neil Holbert (marketing contexts? Are there alternative explanations for why your brain 
researcher) caution, “Is there evidence that the brain activity area might be ‘lighting up’?” And one study showed that only 
co-occurs with the psychological process [the researcher] is 16 percent of respondents felt comfortable allowing their brain to 
interested in under a variety of conditions and in a variety of be scanned for research. 


Directness of Observation 


Direct observation occurs when the observer is physically present and personally monitors what takes 
place. This approach is very flexible because it allows the observer to react to and report subtle aspects 
of events and behaviors as they occur. He or she is also free to shift places, change the focus of the 
observation, or concentrate on unexpected events if they occur. A weakness of this approach is that ob- 
servers’ perception circuits may become overloaded as events move quickly, and observers must later 
try to reconstruct what they were not able to record. Also, observer fatigue, boredom, and distracting 
events can reduce the accuracy and completeness of observation. 

Indirect observation occurs when the recording is done by mechanical, photographic, or electronic 
means. For example, a special camera that takes one frame every second may be mounted in a depart- 
ment of a large store to study customer and employee movement. Indirect observation is less flexible 
than direct observation but is also much less biasing and may be less erratic in accuracy. Another ad- 
vantage of indirect observation is that the permanent record can be reanalyzed to include many different 
aspects of an event. Electronic recording devices, which have improved in quality and declined in cost, 
are being used more frequently in observation research. 


Concealment 


A second factor affecting the observer-participant relationship concerns whether the participant should 
know of the observer’s presence. When the observer is known, there is a risk of atypical activity by 
the participant. The initial entry of an observer into a situation often upsets the activity patterns of the 
participants, but this influence usually dissipates quickly, especially when participants are engaged 
in some absorbing activity or the presence of observers offers no potential threat to the participants’ 
self-interest. The potential bias from participant awareness of observers is always a matter of concern, 
however. 
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Observers use concealment to shield themselves from the object of their observation. Often, techni- 
cal means such as one-way mirrors, hidden cameras, or microphones are used. These methods reduce 
the risk of observer bias but bring up a question of ethics. Hidden observation is a form of spying, and 
the propriety of this action must be reviewed carefully. 

A modified approach involves partial concealment. The presence of the observer is not concealed, 
but the objectives and participant of interest are. A study of selling methods may be conducted by 
sending an observer with a salesperson who is making calls on customers. However, the observer’s 
real purpose may be hidden from both the salesperson and the customer (e.g., she may pretend she is 
analyzing the display and layout characteristics of the stores they are visiting). 


Participation 


The third observer-participant issue is whether the observer should participate in the situation 
while observing. A more involved arrangement, participant observation, exists when the ob- 
server enters the social setting and acts as both an observer and a participant. Sometimes he or she 
is known as an observer to some or all of the participants; at other times the true role is concealed. 
While reducing the potential for bias, this again raises an ethical issue. Often participants will not 
have given their consent and will not have knowledge of or access to the findings. After being 
deceived and having their privacy invaded, what further damage could come to the participants if 
the results became public? This issue needs to be addressed when concealment and covert partici- 
pation are used. 

Participant observation makes a dual demand on the observer. Recording can interfere with par- 
ticipation, and participation can interfere with observation. The observer’s role may influence the way 
others act. Because of these problems, participant observation is used less in business research than, 
say, in anthropology or sociology. It is typically restricted to cases in which nonparticipant observation 
is not practical—for example, a study of the functioning of a traveling auditing team. 


> Conducting an Observation Study 


The Type of Study 


Observation is found in almost all research studies, at least at the exploratory stage. Such data col- 
lection is known as simple observation. Its practice is not standardized, as one would expect, be- 
cause of the discovery nature of exploratory research. The decision to use observation as the major 
data collection method may be made as early as the moment the researcher moves from research 
questions to investigative questions. The latter specify the outcomes of the study—the specific ques- 
tions the researcher must answer with collected data. If the study is to be something other than 
exploratory, systematic observation employs standardized procedures, trained observers, sched- 
ules for recording, and other devices for the observer that mirror the scientific procedures of other 
primary data methods. Systematic studies vary in the emphasis placed on recording and encoding 
observational information: 


At one end of the continuum are methods that are unstructured and open-ended. The observer tries to provide 

as complete and nonselective a description as possible. On the other end of the continuum are more structured 
and predefined methods that itemize, count, and categorize behavior. Here the investigator decides beforehand 
which behavior will be recorded and how frequently observations will be made. The investigator using structured 
observation is much more discriminating in choosing which behavior will be recorded and precisely how [it is] to 
be coded.* 


One author classifies observation studies by the degree of structure in the environmental setting 
and the amount of structure imposed on the environment by the researcher,* as reflected in Ex- 
hibit 8-4. The researcher conducting a class |, completely unstructured study would be in a natural 
or field setting endeavoring to adapt to the culture. A typical example would be an ethnographic 
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>Exhibit 8-4 Classification of Observation Studies 


Research Class Environment Purpose Research Tool 

1. Completely unstructured Natural setting Generate hypotheses 

2. Unstructured Laboratory | 

8. Structured Natural setting Observation checklist 


4. Completely structured Laboratory Test hypotheses Observation checklist 


study in which the researcher, as a participant-observer, becomes a part of the culture and describes 
in great detail everything surrounding the event or activity of interest. Donald Roy, in the widely 
used case in organizational behavior, “Banana Time,” took a punch press job in a factory to describe 
the rituals that a small work group relied on to make their highly repetitive, monotonous work bear- 
able.® With other purposes in mind, business researchers may use this type of study for hypothesis 
generation. 

Class 4 studies—completely structured research—are at the opposite end of the continuum from 
completely unstructured field investigations. The research purpose of class 4 studies is to test hy- 
potheses; therefore, a definitive plan for observing specific, operationalized behavior is known in 
advance. This requires a measuring instrument, called an observation checklist, which is analogous 
to a questionnaire. Exhibit 8-5 shows the parallels between survey design and checklist development. 


>Exhibit 8-5 Flowchart for Observation Checklist Design 
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Checklists should possess a high degree of precision in defining relevant behavior or acts and have 
mutually exclusive and exhaustive categories. The coding is frequently closed, thereby simplifying 
data analysis. The participant groups being observed must be comparable and the laboratory condi- 
tions identical. The classic example of a class 4 study was Bales’s investigation into group interac- 
tion.’ Many team-building, decision-making, and assessment center studies follow this structural 
pattern. 

The two middle classes of observation studies emphasize the best characteristics of either 
researcher-imposed controls or the natural setting. In class 2, the researcher uses the facilities of a 
laboratory—videotape recording, two-way mirrors, props, and stage sets—to introduce more control 
into the environment while simultaneously reducing the time needed for observation. In contrast, a 
class 3 study takes advantage of a structured observational instrument in a natural setting. 


Content Specification 


Specific conditions, events, or activities that we want to observe determine the observational re- 
porting system (and correspond to measurement questions). To specify the observation content, we 
should include both the major variables of interest and any other variables that may affect them. 
From this cataloging, we then select those items we plan to observe. For each variable chosen, 
we must provide an operational definition if there is any question of concept ambiguity or special 
meanings. Even if the concept is a common one, we must make certain that all observers agree 
on the measurement terms by which to record results. For example, we may agree that variable W 
will be reported by count, while variable Y will be counted and the effectiveness of its use judged 
qualitatively. 

Observation may be at either a factual or an inferential level. Exhibit 8-6 shows how we could 
separate the factual and inferential components of a salesperson’s presentation. This table is sug- 
gestive only. It does not include many other variables that might be of interest, including data 
on customer purchase history; company, industry, and general economic conditions; the order in 
which sales arguments are presented; and specific words used to describe certain product charac- 
teristics. The particular content of observation will also be affected by the nature of the observa- 
tion setting. 


>Exhibit 8-6 Content of Observation: Factual versus Inferential 


Factual 


Inferential 


Introduction/identification of salesperson and customer. Credibility of salesperson. Qualified status of customer. 


Product presented. Customer interest in product. 


Number of customer objections raised per product. Customer concerns about features and benefits. 


Salesperson’s attempt to restore controls. 


Effectiveness of salesperson’s control attempt. 
Consequences for customer who prefers interaction. 


Environmental factors interfering with the interview. Level of distraction for the customer. 
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Observer Training 


There are a few general guidelines for the qualification and selection of observers: 


¢ Concentration: Ability to function in a setting full of distractions. 
¢ Detail-oriented: Ability to remember details of an experience. 
¢ Unobtrusive: Ability to blend with the setting and not be distinctive. 


e Experience level: Ability to extract the most from an observation study. 


An obviously attractive observer may be a distraction in some settings but ideal in others. The same can 
be said for the characteristics of age or ethnic background. 

If observation is at the surface level and involves a simple checklist or coding system, then 
experience is less important. Inexperience may even be an advantage if there is a risk that expe- 
rienced observers may have preset convictions about the topic. Regardless, most observers are 
subject to fatigue, halo effects, and observer drift, which refers to a decay in reliability or validity 
over time that affects the coding of categories.’ Only intensive videotaped training relieves these 
problems. 

The observers should be thoroughly versed in the requirements of the specific study. Each observer 
should be informed of the outcomes sought and the precise content elements to be studied. Observer 
trials with the instrument and sample videotapes should be used until a high degree of reliability is ap- 
parent in their observations. When there are interpretative differences between observers, they should 
be reconciled. 


Data Collection 


The data collection plan specifies the details of the task. In essence it answers the questions who, what, 
when, how, and where. 


Who? 


What qualifies a participant to be observed? Must each participant meet a given criterion—those who 
initiate a specific action? Who are the contacts to gain entry (in an ethnographic study), the intermedi- 
ary to help with introductions, the contacts to reach if conditions change or trouble develops? Who 
has responsibility for the various aspects of the study? Who fulfills the ethical responsibilities to the 
participants? 


What? 


The characteristics of the observation must be set as sampling elements and units of analysis. This is 
achieved when event-time dimension and “act” terms are defined. In event sampling, the researcher 
records selected behavior that answers the investigative questions. In time sampling, the researcher 
must choose among a time-point sample, continuous real-time measurement, or a time-interval sample. 
For a time-point sample, recording occurs at fixed points for a specified length. With continuous mea- 
surement, behavior or the elapsed time of the behavior is recorded. Like continuous measurement, 
time-interval sampling records every behavior in real time but counts the behavior only once during 
the interval.’ 

Assume the observer is instructed to observe a quality control inspection for 10 minutes out of 
each hour (a duration of two minutes each for five times). Over a prolonged period, if the samples are 
drawn randomly, time sampling can give a good estimate of the pattern of activities. In a time-interval 
sampling of workers in a department, the outcome may be a judgment of how well the department is 
being supervised. In a study of sales presentations using continuous real-time sampling, the research 
outcome may be an assessment of a given salesperson’s effectiveness or the effectiveness of different 
types of persuasive messages. 
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Walmart Boosts RFID Technology for Observation 


A consortium of 36 consumer packaged-goods (CPG) manu- 
facturers, research companies, and universities have been 
working to change the way consumers generate and deliver 
purchase and consumption information, as well as how this 
information is integrated with detailed supply chain manage- 
ment information. If the Auto ID Center gets its way, all future 
CPGs will contain radio frequency identification (RFID) smart 
abels that will send signals to Internet databases and track a 
specific product unit from manufacturing through warehous- 
ing, retail display, and storage and potentially through con- 
sumer storage and consumption and the recycling center. 
Walmart recently gave the technology a boost by indicating 
it would put the radio tags on its ready-to-wear in more than 
8750 stores in the U.S. By its sheer size, Walmart’s dictate 
could transform how observation studies in CPG are done. 
Goodyear, the world’s largest tire manufacturer, also an- 
nounced that its tires shipped to Walmart would contain an 
RFID microchip within the tire itself, which could also relay 
tire information to drivers about inflation and wear to improve 
vehicle safety. 

From a research perspective, the opportunities seem 
enormous. Store shelf and display locations could be evalu- 
ated for promotional effectiveness, especially when combined 
with shopper card information. Average age of inventory 
could be known to the day or hour by matching the RFID 
location information with shipping and receiving documents. 
Product recalls could be handled with efficiency. While bar 


Other important dimensions are defined by acts. What constitutes an act is established by the needs 
of the study. It is the basic unit of observation. Any of the following could be defined as an act for an 


observation study: 


e A single expressed thought. 
e A physical movement. 

° A facial expression. 

e A motor skill. 


codes currently provide similar information to that promised 
by RFID tags, they must be scanned to be read. RFID tags 
need no such intervention but transmit continuously until dis- 
abled. Technically, if the tag on the product is not disabled at 
the time of purchase, the RFID tag could transmit location in- 
formation from a consumer’s car, home, or refrigerator. Such 
signals broadcast from a consumer’s home or car would re- 
quire compliance, similar to the way consumers volunteer to 
be part of ACNielsen’s Homescan panel today. But accord- 
ing to Katherine Albrecht, founder of consumer action group 
CASPIAN, “Supermarket cards and retail surveillance devices 
are merely the opening volley of the marketers’ war against 
consumers. If consumers fail to oppose these practices now, 
our long-term prospects may look like something from a dys- 
topian science fiction novel.” 

The main driver of the RFID movement is supply chain 
savings. According to one analyst, Walmart could save bil- 
lions by perfecting its inventory management with the infor- 
mation provided by the tags. It also has the possibility of 
reducing theft, which is another large savings. Walmart has 
climbed to the top of the retailing world by reducing its op- 
eration costs and using those savings to reduce retail prices 
for customers. 


http://walmartstores.com; www.goodyear.com; 
www.nocards.org; www.nielsen.com; 
http://trolleyscan.com 


Although acts may be well defined, they often present difficulties for the observer. A single state- 
ment from a sales presentation may include several thoughts about product advantages, a rebuttal to 
an objection about a feature, or some remark about a competitor. The observer is hard-pressed to sort 
out each thought, decide whether it represents a separate unit of observation, and then record it quickly 
enough to follow continued statements. 


The Eyes Have It 
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Researchers have been using eye-tracking studies for more than 
100 years to understand what people see when presented with vi- 
sual stimuli. Current online surveys often present visual stimuli, but 
such surveys only obtain data about what the participant recalls or 
intends. There is a significant gap between what people say they 
saw and what they actually saw, shared Ephraim (Jeff) Bander, 
president North America—general manager at EyeTrackShop. 
Eye-tracking studies reveal where the eye rests (fixates). Tradi- 
tional eye-tracking studies involved a lab setting with expensive 
equipment. As a result, such studies until recently were restricted 
to a relatively elite group of client users. For many potential clients, 
the cost was too high, turn around was too slow, and too small 
samples were restricted geographically due to the need to be near 
a lab where tracking hardware and software was located. Swed- 
ish firm EyeTrackShop, Inc. is changing all these parameters. 
Using a participant’s own laptop or tablet Web camera, and 
calibrating it via a Web-based exercise (where the participant sits 
in front of their Web camera and follows a moving blue spot on 
their screen), EyeTrackShop can now reach a worldwide audi- 
ence, in a fraction of the time and cost of established eye-tracking 
studies. “For example” shared Bander, “P&G completed a study 
in four countries with 400 people; the study was completed in less 
than a week at a fraction of the cost for air travel alone.” It’s also 
about one-fifth the cost of a traditional eye-tracking study. The 
starting cost of EyeTrackShop’s single-country study is $5,000. 
EyeTrackShop uses a sample provider to recruit partici 
pants or a client might want to use its own panel. “Anything that 
can be presented on a screen can be tested,” shared Bander. 


EyeTrackShop has tested print ads, packaging designs, shelf 
sets for stores, as well as TV, online video, Web pages and on- 
line ads. A typical project, for example the test of a YouTube ad, 
involves asking permission of the participant to calibrate his or 
her webcam on their computer, the presentation of the stimulus 
to be tracked (e.g., a display ad on YouTube), the gathering of 
the “facts” —actual eye movement data—as well as data from an 
online survey (e.g., questions about brand and message aware- 
ness, purchase intention, or any other topic on which the client 
might want data) all culminating in a concise, understandable 
final report. “The questions are designed to reveal the see-say 
gap—the actual people who look at something versus those that 
say they saw something,” explained Bander. 

The reports contain some very interesting features, including a 
Web fixation map (order of page elements where the participant’s 
eye rested), with average time to fixation on each element, and 
a visual attention “heat” map (uses color to show where partici- 
pant was fixated), and a visual attention opacity map (blacks out 
what the participant didn’t see). All client data are compared with 
benchmark data for the given medium of the test stimulus based 
on thousands of studies previously conducted. (See a sample of a 
client report on the text website: EyeTrackShop.pdf.) 

EyeTrackShop operates in 40 countries. In just two years it 
has amassed more than 250,000 eye-tracking studies, serving 
such clients as SnapfFish, AOL, P&G, Google, Reebok, General 
Mills, H&M, GNC, and Clear Channel. 


www.eyetrackshop.com 


When? 


Is the time of the study important, or can any time be used? In a study of out-of-stock conditions in 
a supermarket, the exact times of observation may be important. Inventory is shipped to the store on 
certain days only, and buying peaks occur on other days. The likelihood of a given product being out 
of stock is a function of both time-related activities. 


How? 


Will the data be directly observed? If there are two or more observers, how will they divide the task? 
How will the results be recorded for later analysis? How will the observers deal with various situations 
that may occur—when expected actions do not take place or when someone challenges the observer 
in the setting? 
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When Physicians and Their Patients Are Research Subjects 


When Akron Children’s Hospital asked Marcus Thomas LLC to 
renew its brand promise, Marcus Thomas chose an observation 
study to guide its work. To peel down to the emotional core 
driving parents’ hospital selection decision, Marcus Thomas ne- 
gotiated permission to shadow physicians and parents during 
testing, diagnosis, and treatment of the child patients. 

When physicians and patients are the subjects of research, 
ethical issues abound. But the observation study was “the best 
way to reveal the core emotional experiences and motivations 
for parents’ decisions,” according to Jennifer Hirt-Marchand, 
vice president and director of research for Marcus Thomas. So 
firm researchers spent days in the hospital watching interactions 
among patients, parents, physicians, nurses, and other staff. 
They were careful to respect the sensitivity of the subject and the 
environment they observed. “While parents were unbelievably co- 
operative, the researchers tried to be as unobtrusive as possible,” 


compassion, technical skill, competence, etc.) toward the 


said Robin Segbers, Marcus Thomas’s manager of planning. 
Listening to physician conversations with parents and child 
patients revealed to researchers a core brand promise that 
Akron Children’s Hospital could embody and that would influ- 
ence parents to choose Akron Children’s for acute care (i.e., 
requiring more than three consecutive days of hospitalization): 


simple goal of helping every child reach his or her full potential. 


This observation study was phase one of research that sub- 
sequently included focus groups and telephone surveys. An ad 
campaign built on the findings of this research succeeded in 
increasing occupancy at Akron Children’s by 11 percent. For 
more information, read the case. “Akron Children’s Hospital: 


Akron Children’s Hospital focuses ALL of the hospital’s Part A and Part B.” 


resources (energy, creativity, state-of-the-art technology, www.akronchildrens.org; www.marcusthomasllc.com 


Where? 


Within a spatial confine, where does the act take place? In a retail traffic pattern study, the proximity 
of a customer’s pause space to a display or directional sign might be recorded. Must the observation 
take place in a particular location within a larger venue? The location of the observation, such as a sales 
approach observation within a chain of retail stores, can significantly influence the acts recorded. 
Observers face unlimited variations in conditions. Fortunately, most problems do not occur simultane- 
ously. When the plans are thorough and the observers well trained, observation research is quite successful. 


> Unobtrusive Measures 


Up to this point, our discussion has focused on direct observation as a traditional approach to data collec- 
tion. Like surveys and experiments, some observation studies—particularly participant observation— 
require the observer to be physically present in the research situation. This contributes to a reactivity 
response, a phenomenon in which participants alter their behavior in response to the researcher. 

Webb and his colleagues have given us an insight into some very innovative observational pro- 
cedures that can be both nonreactive and inconspicuously applied. Called unobtrusive measures, 
these approaches encourage creative and imaginative forms of indirect observation, archival searches, 
and variations on simple and contrived observation.'° Of particular interest are measures involving 
indirect observation based on physical traces that include erosion (measures of wear) and accretion 
(measures of deposit). 


Designing the Observation Study 
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The design of a behavioral observation study follows the same 
pattern as other research. Once the researcher has specified 
the investigative questions, it is often apparent that the best way 
to conduct the study is through observation. Guidance for con- 
ducting a behavioral observation and translating the investigative 
question(s) into an observation checklist is the subject of this 
Closeup. We first review the procedural steps and then explain 
how to create a checklist. 

Most studies that use behavioral observation follow a gen- 
eral sequence of steps that parallel the research process. (See 
Exhibit 8-1.) Here we adapt those steps to the terminology of the 
observation method: 


e Define the content of the study. 

e Develop a data collection plan that identifies the 
observational targets, sampling strategy, and acts 
(operationalized as a checklist or coding scheme). 

e Secure and train observers. 

e Collect the data. 

e Analyze the data. 


In this chapter’s Bringing Research to Life, we recount an 
incident in which Sara Arens is subcontracting a behavioral 
observation study to MarketViews, a specialty research firm. 
The client, HomeExtravaganza, is experiencing declining repeat 
visits at its new superstore units. Preliminary results from a peri- 
odic survey indicate that customer confusion in the megastore 
is discouraging customers from returning, so an observation 
study is planned to see how employees and store design ele- 
ments contribute to or solve the problem. The research ques- 
tions might be: 


e What do employees do to reduce or eliminate customer 
confusion? 

e What do employees do that contributes to customer 
confusion? 

e Which design elements diffuse customer confusion? 

e Which design elements contribute to customer confusion? 


Further assume that the survey indicates that customers who 
feel confused and cited this confusion as their reason for not 
returning had entered the store looking for a variety of merchan- 
dise, stocked in various locations throughout the vast store. 
They described their hopelessness as evolving from the experi- 
ence of not knowing where to start the process of searching. 
Such customers showed no particular similarities or differences 
in terms of ethnic background, age, or education level. Some 
had the assistance of greeters or floor assistance associates, 
while others did not. 


The observation targets will be twofold: shoppers entering 
the store through the main entrance and employees serving as 
greeters and floor assistance associates. Customers who re- 
quest assistance from the main-entrance greeter or who consult 
the directional location signs will be the primary target. If they 
approach a floor assistance associate, the employee will also 
become a target. 

Survey research reveals some inconsistency by time of 
day and type of merchandise sought, so MarketViews plans to 
sample during four primary day parts—early morning, midday, 
afternoon, and early evening—as well as at all three “directional 
locations,” where signage describing the store and the shop- 
per’s current “You are here” location is noted. 

Notes taken by Sara during a tour of the store help to 
identify the acts to record. During their subsequent meet- 
ing, it is decided that MarketViews will record the customer's 
seeking of assistance, either personal or signage; his or her 
consulting of the directional signage and its location; the 
customer’s path to the desired merchandise; and whether a 
purchase is completed. The assistance acts of the floor assis- 
tance associate will also be recorded. These are determined 
to be assistance versus no assistance, pointing plus verbal 
direction, verbal direction only, providing the customer with 
a store copy-sheet directional map on which the associate 
marks the location of the merchandise and the path to get 
there, inquiry to other staff for location assistance, passing 
the customer to another floor assistance associate, accom- 
panying the customer to the correct aisle, accompanying the 
customer to actual merchandise shelf location, and providing 
verbal assistance to selection of the appropriate product from 
the array of products provided. 

It is determined that a checklist will be created and tested 
by MarketViews, with any necessary changes occurring after 
the test. The checklist developed is shown in Exhibit 8-7. The 
foremost concern is that either the customer or the associate 
will discover that he or she is being observed and will change 
behavior. Human observers will be used to trace the path of 
observational targets. By means of the store’s security cam- 
eras, researchers will record customers flowing through the 
main entrance and past the greeter location and stopping at 
directional location signs. Counts of brochure store maps dis- 
tributed from the directional signage locations and copysheet 
directional maps used by floor assistance associates will 
also be used as a measure of customers seeking directional 
assistance. 


(continued) 
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>Exhibit 8-7 Sample Checklist for HomeExtravaganza Study 


Time Day: M T W Th F Sa Su Date Store No. Observer # 
Target Customer Interception Location: QO Main Entry OQ Directional Location Sign: O #1 
O #2 
O #3 
Target Shopper Characteristics: Male OU Female 
OChild OChild-teen+ OAdult O Senior 
Shopper Companion(s): QO Alone OWithothers: Uotheradult No..__-M _ No. —__F 
Q child/children No. —_ 
Shopping Cart Used: ONo QO Yes 
Greeter verbal interaction with target! ONo OYes_ Greeter No. 
Action JU Point to directional sign 
QO) Verbal directions 
Floor Assistance Associate Interaction: CUINo OU Yes __ Interception location: Aisle# Crossway# 
Associate # 
Assistance given: UNo JU Yes Action: UO Verbal direction only 
U Verbal direction plus pointing 
QO) Store directional copy-map with marked mdse location 
QO) Store directional copy-map with mdse location & path 
Q) Inquire of other staff 
C) Pass customer to another FAA 
1 Accompany customer to aisle location 
C1 Accompany customer to mdse shelf location 
Q) Product selection assistance offered 
Directional Sign Interaction: C@No OYes_ Signlocation: O #1 
O #2 
O #3 
Purchase: UNo QO Yes: Item Sought Assistance For: ONo JO Yes 
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Natural erosion measures are illustrated by the frequency of replacement of vinyl floor tile in front 
of museum exhibits as an indicator of exhibit popularity. The study of wear and tear on book pages is 
a measure of library book use. Counting the remaining brochures in a car dealer’s display rack after a 
favorable magazine review suggests consumer interest. 

Physical traces also include natural accretion such as discovering the listenership of radio stations 
by observing car radio settings as autos are brought in for service. Another type of unobtrusive study 
involves estimating liquor and magazine consumption by collecting and analyzing family trash. An 
interesting application compared beer consumption reports acquired through interviews with the find- 
ings of sampled trash. If the interview data were valid, the consumption figures for the area were at 
15 percent. However, the validity was questioned when the beer can count from trash supported a 
77 percent consumption rate.!' 

William Rathje is a professor of archaeology at Stanford University and founder of the Garbage 
Project in Tucson. His study of trash, refuse, rubbish, and litter resulted in the subdiscipline that the 
Oxford English Dictionary has termed garbology. By excavating landfills, he has gained insight into 
human behavior and cultural patterns—sometimes sorting the contents of up to 150 coded categories. 
His previous studies have shown that “people will describe their behavior to satisfy cultural expecta- 
tions, like the mothers in Tucson who unanimously claimed they made their baby food from scratch, 
but whose garbage told a very different tale.”’”” 

Physical trace methods present a strong argument for use based on their ability to provide low-cost 
access to frequency, attendance, and incidence data without contamination from other methods or re- 
activity from participants. They are excellent “triangulation” devices for cross-validation. Thus, they 
work well as supplements to other methods. Designing an unobtrusive study can test a researcher’s 
creativity, and one must be especially careful about inferences made from the findings. Erosion results 
may have occurred because of wear factors not considered, and accretion material may be the result of 
selective deposit or survival. 


Ce 


1 Observation is one of the few options available for study- e The reliability of inferences from surface indicators. 
ing records, mechanical processes, lower animals, small ° The problems of quantification and disproportionately 
children, and complex interactive processes. We can gather large records. 


data as the event occurs and can come closer to capturing 
the whole event than with interrogation. On the other hand, 
we have to be present to catch the event or have some 


e The limitation on presenting activities and inferences 
about cognitive processes. 


recording device on the scene to do the job. 5 Wecan classify observation in terms of the observer- 
participant relationship. This relationship may be viewed 
2 Observation includes a variety of monitoring situations that from three perspectives: (1) Is the observation direct or 
cover nonbehavioral and behavioral activities. indirect? (2) Is the observer’s presence known or unknown? 
3 The strengths of observation as a data collection method (3) Is the observer a participant or nonparticipant? 
include: 6 The design of an observation study follows the same gen- 
© Securing information about people or activities that eral pattern as other research. Observation studies fall into 
cannot be derived from experiments or surveys. four general types based on the degree of structure and 


the nature of the observational environment. The researcher 


e Avoiding participant filtering and forgetting. 
Poe filles aie must define the content of the study; develop a data col- 


° Securing environmental context information. lection plan that identifies participants, sampling strategy, 
e Optimizing the naturalness of the research setting. and “acts” (often operationalized as a checklist or coding 
e Reducing obtrusiveness. scheme); secure and train observers; and launch the study. 


Unobtrusive measures offer an unusual and creative ap- 


4 Observation may be limited by: 
aoe z si proach to reducing reactivity in observation research by in- 


° The difficulty of waiting for long periods to capture the direct observation and other methods. Measures of erosion 
relevant phenomena. and accretion serve as ways to confirm the findings from 
e The expense of observer costs and equipment. other methods or operate as singular data sources. 
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concealment 178 

direct observation 177 
event sampling 181 
extralinguistic behavior 175 
indirect observation 177 
linguistic behavior 175 


nonverbal behavior 175 


observation 173 

observation checklist 179 
observer drift 181 

participant observation 178 
physical condition analysis 174 
physical traces 184 


process (activity) analysis 174 


reactivity response 184 
record analysis 173 

simple observation 178 
spatial relationships 175 
systematic observation 178 
time sampling 181 
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Terms in Review 


{ 


Compare the advantages and disadvantages of the survey 
to those of observation. Under which circumstances could 
you make a case for using observation? 


What ethical risks are involved in observation? In the use of 
unobtrusive measures? 


Based on present or past work experience, suggest prob- 


lems that could be resolved by using observation-based 
data. 


4 Distinguish between the following: 


a_ The relative value of communication and observation. 
b Nonverbal, linguistic, and extralinguistic analysis. 


c Factual and inferential observation. 


Making Research Decisions 
5 The observer—participant relationship is an important con- 


sideration in the design of observation studies. What kind 

of relationship would you recommend in each of the follow- 

ing cases? 

a Observations of professional conduct in the classroom 
by the student author of a course evaluation guide. 


b Observation of retail shoppers by a researcher who is 
interested in determining customer purchase time by 
type of goods purchased. 


c Observation of a focus group interview by a client. 


d Effectiveness of individual farmworker organizers in their 
efforts to organize employees of grape growers. 


Assume you are the manufacturer of modular office sys- 
tems and furniture as well as office organization elements 
(desktop and wall organizers, filing systems, etc.). Your 
company has been asked to propose an observation study 
to examine the use of office space by white-collar and 
managerial workers for a large insurance company. This 


study will be part of a project to improve office efficiency 
and paperwork flow. It is expected to involve the redesign 
of office space and the purchase of new office furniture and 
organization elements. 


a What are the varieties of information that might be 
observed? 


b Select a limited number of content areas for study, and 
operationally define the observation acts that should be 
measured. 


c Develop a checklist to be used by observers in the pre- 
vious study. Determine how many observers you need, 
and assign two or three to a specific observation task. 


d Compare the results of your group members’ checklists 
for stability of recorded perceptions. 


Amazon's Kindle, its electronic book, magazine, and news- 
paper reader, has been far more successful than its SONY 
competitor. This could be at least somewhat influenced by 
the “Oprah factor”; the Kindle was first introduced on the 
Oprah show and strongly endorsed by the mega-mogul. 
But the original Kindle had some physical problems—button 
placement and page-turn speed among them. Correcting 
these problems resulted in the Kindle2, released in 2009 

to glowing reviews, and ultimately to the Kindle Fire. What 
observation research might have been used in the redesign 
of the original Kindle? 


You wish to analyze the pedestrian traffic that passes a 
given store in a major shopping center. You are interested 
in determining how many shoppers pass by this store, 

and you would like to classify these shoppers on various 
relevant dimensions. Any information you secure should be 
obtainable from observation alone. 


a What other information might you find useful to observe? 


b How would you decide what information to collect? 


c Devise the operational definitions you would need. 
d What would you say in your instructions to the observers 
you plan to use? 
e How might you sample this shopper traffic? 
Bringing Research to Life 


9 Develop the investigative questions that should have 
guided Sara’s observation study of HomeExtravaganza. 


From Concept to Practice 
10 Using Exhibit 8-3, identify the type of study described in 
each of the Snapshots featured in this chapter. 
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From the Headlines 

11. Walmart got a very mixed reaction when it announced that 
after 30 years it was eliminating greeters from its overnight 
shift in more than 3800 stores, redefining the role of day- 
shift greeters, and moving them further into the store and 
away from the door. What observation research would you 
have conducted to reach these decisions? 


ee 


& Akron Children’s Hospital 


ee Envirosell 


Net Conversions Influence Kelley Blue Book 


State Farm: Dangerous Intersections 


* You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 
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learningobjectives 


After reading this chapter, you should understand . . . 


The uses for experimentation. 

The advantages and disadvantages of the experimental method. 
The seven steps of a well-planned experiment. 

Internal and external validity with experimental research designs. 


The three types of experimental designs and the variations of each. 


6 6 This is a time of [e-book app] experimentation. 
I’d be really disappointed if we weren’t seeing 
both successes and failures. | do think that 


everyone seems to be looking at these 
differently. 9 9 


Barbara Marcus, consultant and adviser, 
Open Road Integrated Media 


bringingresearcnhtolife 


Jason slides into a seat next to Sara at the conference 
lunch table. Sara glances his way and nods briefly but 
keeps her attention on the man to her left. He is describ- 
ing some of the finer details of the Point of Purchase 
Advertising Institute’s groundbreaking experiment to 
put display materials on a directly comparative basis 
with other audited advertising and sales promotion 
activities. Sara had just attended his presentation dur- 
ing the morning session of the conference. 

As he stops speaking, Sara introduces Jason, “Doug 
Adams, I’d like you to meet my colleague, Jason 
Henry. Jason, Doug is vice president and cofounder of 
Prime Consulting Group, Inc. [Prime].” 

Jason extends his hand across Sara’s plate, “Pleased 
to meet you, Doug. Prime took the lead on that POP 
[point-of-purchase] industry experiment, right? Sorry 
I missed your presentation, but Sara,’ Jason nods to 
Sara, “suggested we divide and conquer—to cover 
more sessions.” 

Doug returns Jason’s smile and handshake. “It 
seems to be the topic of conversation at this table at the 
moment, so maybe you’ll get some of the content here,” 
welcomes Doug. “I was just explaining how we needed 
a methodology that could separate out other sales 
influencers, like price, local advertising, media-delivered 
coupons, or a secondary stocking location in order to 
measure the sales lift generated by the POP material. 
For example, if Frito-Lay offered Doritos at $2.49, 
reduced from $2.99, plus it stocked a secondary location 
near the soft-drink aisle, Frito-Lay could track the sales 
lift. If sales increased even more when a Doritos sign 
was posted over the secondary stock location, then the 
power of the POP could be determined.” 

“Your session was generating all the buzz as I 
entered the dining room,” comments Jason. “How 
many types of POP were assessed?” 


“Ultimately, 20 different types,” shares Doug. 
“Several different message types—for example, brand 
name, photo, price, retail savings, thematic .. . like a 
movie tie-in... or generic . . . like the summer barbeque 
season—and numerous locations.” 

“Like regular shelf stocking location, end-cap, front 
lobby .. . ?” asks Jason. At Doug’s affirmative nod 
Jason asks, “How did you keep the manufacturers from 
distorting the experiment?” 

To give Doug a chance to take a bite, Sara supplies, 
“Prime used a double-blind audit tracking procedure 
using observation, coupled with sales tracking 
through more than 250 supermarkets from the IRI 
panel of stores and 120 convenience stores from six 
retailers with Nielsen’s Market Decisions program. 
The manufacturers didn’t know which stores were 
involved.” 

“And did the POP create the sales lift the retailers 
expected?” asks Jason. 

“Not only were we able to calculate sales lift for each 
type of promotion,” shares Doug, “but we were also able 
to calculate a full cost-per-thousand [CPM] estimate, 
including the cost of manufacturing, delivering, and 
installing the point-of-purchase material. TV, radio, and 
in-store ads are still quoting CPM exposures without 
the cost of the ad.” 

“If grocery and convenience stores buy in, this will 
be a large piece of business for one or several firms,” 
comments Sara. “What’s been the reaction?” 

“When a retailer, who has been relying on his gut 
instinct to accept or reject POP materials for his store, 
sees that the same material can cause a 20 to 40 percent 
lift in store sales, he’s bound to be receptive. We think 
it’s going to be a major new research initiative,” smiles 
Doug. “Are you interested?” 
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Is Experimentation? 


Why do events occur under some conditions and not under others? Research methods that answer 
such questions are called causal methods. (Recall the discussion of causality in Chapter 6.) Ex post 
facto research designs, in which a researcher interviews respondents or observes what is or what 
has been, also have the potential for discovering causality. The distinction between these methods 
and experimentation is that the researcher is required to accept the world as it is found, whereas an 
experiment allows the researcher to alter systematically the variables of interest and observe what 
changes follow. 

In this chapter we define experimentation and discuss its advantages and disadvantages. An 
outline for the conduct of an experiment is presented as a vehicle to introduce important con- 
cepts. The questions of internal and external validity are also examined: Does the experimental 
treatment determine the observed difference, or was some extraneous variable responsible? And 
how can one generalize the results of the study across times, settings, and persons? The chapter 
concludes with a review of the most widely accepted designs, a section on test markets, and a 
Closeup example. 

Experiments are studies involving intervention by the researcher beyond that required for measure- 
ment. The usual intervention is to manipulate some variable in a setting and observe how it affects the 
subjects being studied (e.g., people or physical entities). The researcher manipulates the independent 
or explanatory variable and then observes whether the hypothesized dependent variable is affected by 
the intervention. 

An example of such an intervention is the study of bystanders and thieves.' In this experiment, 
participants were asked to come to an office where they had an opportunity to see a person steal 
some money from a receptionist’s desk. A confederate of the experimenter, of course, did the 
stealing. The major hypothesis concerned whether people observing a theft will be more likely 
to report it (1) if they are alone when they observe the crime or (2) if they are in the company of 
someone else. 

There is at least one independent variable (IV) and one dependent variable (DV) in a causal 
relationship. We hypothesize that in some way the IV “causes” the DV to occur. The independent or 
explanatory variable in our example was the state of either being alone when observing the theft or 
being in the company of another person. The dependent variable was whether the subjects reported 
observing the crime. The results suggested that bystanders were more likely to report the theft if they 
observed it alone rather than in another person’s company. 

On what grounds did the researchers conclude that people who were alone were more likely 
to report crimes observed than people in the company of others? Three types of evidence form 
the basis for this conclusion. First, there must be an agreement between independent and de- 
pendent variables. The presence or absence of one is associated with the presence or absence of 
the other. Thus, more reports of the theft (DV) came from lone observers (IV ,) than from paired 
observers (IV,). 

Second, beyond the correlation of independent and dependent variables, the time order of the occur- 
rence of the variables must be considered. The dependent variable should not precede the independent 
variable. They may occur almost simultaneously, or the independent variable should occur before the 
dependent variable. This requirement is of little concern since it is unlikely that people could report a 
theft before observing it. 

The third important support for the conclusion comes when researchers are confident that other 
extraneous variables did not influence the dependent variable. To ensure that these other variables are 
not the source of influence, researchers control their ability to confound the planned comparison. Under 
laboratory conditions, standardized conditions for control can be arranged. The crime observation 
experiment was carried out in a laboratory set up as an office. The entire event was staged without the 
observers’ knowledge. The receptionist whose money was to be stolen was instructed to speak and act 
in a specific way. Only the receptionist, the observers, and the “criminal” were in the office. The same 
process was repeated with each trial of the experiment. 

Although such controls are important, further precautions are needed so that the results achieved 
reflect only the influence of the independent variable on the dependent variable. 
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> An Evaluation of Experiments 
Advantages 


When we elaborated on the concept of cause in Chapter 6, we said causality could not be proved with 
certainty but the probability of one variable being linked to another could be established convincingly. 
The experiment comes closer than any primary data collection method to accomplishing this goal. The 
foremost advantage is the researcher’s ability to manipulate the independent variable. Consequently, 
the probability that changes in the dependent variable are a function of that manipulation increases. 
Also, a control group serves as a comparison to assess the existence and potency of the manipulation. 

The second advantage of the experiment is that contamination from extraneous variables can be 
controlled more effectively than in other designs. This helps the researcher isolate experimental vari- 
ables and evaluate their impact over time. Third, the convenience and cost of experimentation are supe- 
rior to other methods. These benefits allow the experimenter opportunistic scheduling of data collection 
and the flexibility to adjust variables and conditions that evoke extremes not observed under routine 
circumstances. In addition, the experimenter can assemble combinations of variables for testing rather 
than having to search for their fortuitous appearance in the study environment. 

Fourth, replication—repeating an experiment with different subject groups and conditions—leads 
to the discovery of an average effect of the independent variable across people, situations, and times. 
Fifth, researchers can use naturally occurring events and, to some extent, field experiments (a study 
of the dependent variable in actual environmental conditions) to reduce subjects’ perceptions of the 
researcher as a source of intervention or deviation in their everyday lives. 


Disadvantages 


The artificiality of the laboratory is arguably the primary disadvantage of the experimental method. 
However, many subjects’ perceptions of a contrived environment can be improved by investment in 
the facility. Second, generalization from nonprobability samples can pose problems despite random 
assignment. The extent to which a study can be generalized from college students to managers or ex- 
ecutives is open to question. And when an experiment is unsuccessfully disguised, volunteer subjects 
are often those with the most interest in the topic. Third, despite the low costs of experimentation, 
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Research shows they drink 22 cases of beer each summer. 
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Now let's talk about software support. If the software hiccups or takes an unscheduled break, the 
wrong questions can get answered. Only nonstop support can keep that from happening. 
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Online Dating Industry Claims Vs. Kissing a Lot of Frogs 


The 2012 Marketdata Enterprises U.S. Dating Industry Study 
estimates that the dating services market is a $2.1 billion busi- 
ness in the U.S. Online services capture about 53 percent 
of that total. With the U.S. market saturated with more than 
1500 websites, and competition from popular social network- 
ing sites (Yahoo and Facebook), Europe is seen as the next 
untapped market. 


Revenue Value of Dating Websites (in Billions) 
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From the Chemistry.com website to actual chemical test- 
ing, people seek compatibility matches using methods such as 
hour-long surveys and even DNA samples in order to identify 
their genetic soul mate. Each of several popular Internet dating 
sites claims that its method is more scientifically valid than the 
competition. The proof of effectiveness, however, should make 
research methods students cringe. 

Chemistry.com’s Helen Fisher studied the neural receptors 
of people in love and linked her results to six specific chemicals 
that were subsequently represented on a 56-item questionnaire. 
Despite a 28,000-person sample drawn from its website, there 
was no compelling evidence beyond the old adage that those 


of like mind (having the same goals) have a better chance of 
attracting something permanent. 

Markus Frind, chief executive and founder of Plenty of Fish.com, 
says his site produces 800,000 relationships each year, but 
creates about 100,000 marriages during that time—a figure based 
on “some study | found online.” He adds, “I! don’t want to pay 
$200,000 to a research company to find out how many marriages 
| have per year.” The site’s psychologist, James Houran, claims 
that those meeting through the site were more satisfied with their 
relationship than those using any other system. 

Historically, Match.com counted member self-reports of mar- 
riages. Later it claimed in press releases that it “is credited with 
more marriages than any other site.” This was based on the results 
of a survey of 4,743 couples registered on WeddingChannel.com 
for whom Match.com was the primary connection. Match.com’s 
media kit also claimed 12 marriages per day have their roots with 
the site. The company now says that was imprecise. 

eHarmony, known for its patented 258-item questionnaire, 
chose Harris Interactive to research some of the most interesting 
claims. Harris Interactive estimates that 236 people a day married as 
a result of the site. Using these data and Centers for Disease Con- 
trol and Prevention (CDC) and Census Bureau comparisons, and 
adjusting for those married in the 20 to 54 age group, eHarmony 
claimed credit for 2 percent of marriages in the country. eHarmony 
also claimed that couples who met through its site were “happier” 
than those who met in other ways (a control group). Psychology pro- 
fessor Jeffrey Lohr and two psychology graduates at the University 
of Arkansas challenged this finding with evidence that eHarmony 
couples had been married only six months (honeymoon effect), 
whereas the control group had been married about two years. 

“You have to kiss a lot of frogs before you find your handsome 
prince,” may continue to be equally valid advice if the online dating 
industry does not use more rigorous methods to justify its claims. 


www.chemistry.com; www.match.com; 
www.PlentyofFish.com; www.eharmony.com 


many applications of experimentation far outrun the budgets for other primary data collection methods. 
Fourth, experimentation is most effectively targeted at problems of the present or immediate future. 
Experimental studies of the past are not feasible, and studies about intentions or predictions are diffi- 
cult. Finally, management research is often concerned with the study of people. There are limits to the 
types of manipulation and controls that are ethical. 


> Conducting an Experiment? 


In a well-executed experiment, researchers must complete a series of activities to carry out their craft 
successfully. Although the experiment is the premier scientific methodology for establishing causation, 
the resourcefulness and creativeness of the researcher are needed to make the experiment live up to its 
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potential. In this section, and as we introduce Exhibit 9-1, we discuss seven activities the researcher 
must accomplish to make the endeavor successful: 

. Select relevant variables. 

. Specify the treatment levels. 

. Control the experimental environment. 

. Choose the experimental design. 

. Select and assign the subjects. 


. Pilot test, revise, and test. 
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. Analyze the data. 


Selecting Relevant Variables 


Throughout the book we have discussed the idea that a research problem can be conceptualized as 
a hierarchy of questions starting with a management problem. The researcher’s task is to translate 
an amorphous problem into the question or hypothesis that best states the objectives of the research. 
Depending on the complexity of the problem, investigative questions and additional hypotheses can 
be created to address specific facets of the study or data that need to be gathered. Further, we have 
mentioned that a hypothesis is a relational statement because it describes a relationship between two 
or more variables. It must also be operationalized, a term we used earlier in discussing how concepts 
are transformed into variables to make them measurable and subject to testing. 
Consider the following research question as we work through the seven points listed above: 


Does a sales presentation that describes product benefits in the introduction of the message lead to improved reten- 
tion of product knowledge? 


Since a hypothesis is a tentative statement—a speculation—about the outcome of the study, it might 
take this form: 


Sales presentations in which the benefits module is placed in the introduction of a 12-minute message produce better 
retention of product knowledge than those where the benefits module is placed in the conclusion. 


The researchers’ challenges at this step are to: 


1. Select variables that are the best operational representations of the original concepts. 
2. Determine how many variables to test. 


3. Select or design appropriate measures for them. 


The researchers would need to select variables that best operationalize the concepts sales presenta- 
tion, product benefits, retention, and product knowledge. The product’s classification and the nature 
of the intended audience should also be defined. In addition, the term better could be operationalized 
statistically by means of a significance test. 

The number of variables in an experiment is constrained by the project budget, the time allocated, 
the availability of appropriate controls, and the number of subjects being tested. For statistical reasons, 
there must be more subjects than variables. 

The selection of measures for testing requires a thorough review of the available literature and in- 
struments. In addition, measures must be adapted to the unique needs of the research situation without 
compromising their intended purpose or original meaning. 


Specifying Treatment Levels 


In an experiment, participants experience a manipulation of the independent variable, called the 
experimental treatment. The treatment levels of the independent variable are the arbitrary or natural 
groups the researcher makes within the independent variable of an experiment. For example, if salary is 
hypothesized to have an effect on employees’ exercising of stock purchase options, it might be divided 
into high, middle, and low ranges to represent three levels of the independent variable. 
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>Exhibit 9-1 Experimentation in the Research Process 


Exploration Exploration 


Experimental 
Research Design 


Data Collection Design 


Sampling Strategy 


Select participants 


Assign participants to 
treatment levels 


Research Reporting 


Management 
Decision 
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>Exhibit 9-2 Experiment of Placement of Benefits Module within Sales Presentation 
Hypothesis: Sales presentations in which the benefits module is placed in the introduction of a 
12-minute message produce better retention of product knowledge by the customer than those 


in which the benefits module is placed in the conclusion. 


Effect = DV, — DV, 


DV, = 


Introduction Body of Sales Presentation Conclusion 


IV, = Benefits Product 
2 module Knowledge 


The levels assigned to an independent variable should be based on simplicity and common sense. 
In the sales presentation example, the experimenter should not select 8 minutes and 10 minutes as the 
starting points to represent the two treatment levels if the average message about the product is 12 min- 
utes long. Similarly, if the benefits module is placed in the first and second minutes of the presentation, 
observable differences may not occur because the levels are too close together. Thus, in the first trial, 
the researcher is likely to position the midpoint of the benefits module the same interval from the end 
of the introduction as from the end of the conclusion (see Exhibit 9-2). 

Under an entirely different hypothesis, several levels of the independent variable may be needed to 
test order-of-presentation effects. Here we use only two. Alternatively, a control group could provide 
a base level for comparison. The control group is composed of subjects who are not exposed to the 
independent variable(s), in contrast to those who receive the experimental treatment. 


Controlling the Experimental Environment 


In our sales presentation experiment, extraneous variables can appear as differences in age, gender, 
race, dress, communications competence, and many other characteristics of the presenter, the message, 
or the situation. These have the potential for distorting the effect of the treatment on the dependent vari- 
able and must be controlled or eliminated. However, at this stage, we are principally concerned with 
environmental control, holding constant the physical environment of the experiment. The introduc- 
tion of the experiment to the subjects and the instructions would likely be videotaped for consistency. 
The arrangement of the room, the time of administration, the experimenter’s contact with the subjects, 
and so forth, must all be consistent across each administration of the experiment. 

Other forms of control involve subjects and experimenters. When subjects do not know if they are 
receiving the experimental treatment, they are said to be blind. When the experimenters do not know if 
they are giving the treatment to the experimental group or to the control group, the experiment is said 
to be double blind. Both approaches control unwanted complications such as subjects’ reactions to 
expected conditions or experimenter influence. 


Choosing the Experimental Design 


Unlike the general descriptors of research design that were discussed in Chapter 6, experimental de- 
signs are unique to the experimental method. They serve as positional and statistical plans to designate 
relationships between experimental treatments and the experimenter’s observations or measurement 
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Does the Subject Line Influence E-mail Open Rates? 


E-mail has become a popular way to communicate with employ- 
ees, customers, suppliers, and other alliance partners, so getting 
those e-mails opened is important business. Ben Chestnut, a 
partner in Rocket Science Group LLC, shares that open rates 
vary widely due to a large number of factors and that companies 
can only track those e-mails that are written in HTML. “When an 
e-mail has an image that is downloaded, we can track the down- 
load. Or if an e-mail contains a link, we can track that someone 
clicked on the link.” Thus, open rates for e-mail studies could 
be understated by excluding those who actually read text-only 
e-mails. 

Rocket Science Group recently ran a study on its e-mail de- 
sign product MailChimp to discover if high-open-rate e-mails had 
significantly different subject lines than low-open-rate e-mails. 
MailChimp offers templates that could be modified if information 
warranted. Chestnut started with campaigns that e-mailed more 
than 100 people (more than 40 million pieces of e-mail). These 


campaigns were sorted by open rates to identify which fit into 
two groups of interest: those with high open rates (60-87 per- 
cent) and those with low open rates (1-14 percent). From each 
group he drew a sample (20) and compared the subject lines. 
“The difference seemed to be in the expectations that were set for 
each e-mail,” explained Chestnut. Promotional offers—discount 
coupons or percentage-off savings—on subject lines were a sig- 
nal that the e-mail was one designed to sell. If the recipient had 
opted in for only informative e-mail (e.g., a newsletter), then they 
would not be expecting or welcome a hard-sell e-mail. Two con- 
clusions drawn from the study included: (1) that e-mails should 
specify the content in the e-mail subject line (surprisingly, not all 
do) and (2) that the mailer should always honor the parameters of 
its negotiated relationship with the intended e-mail receiver. How 
would you evaluate this study? 


www.rocketsciencegroup.com; www.mailchimp.com 
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points in the temporal scheme of the study. In the conduct of the experiment, the researchers apply their 
knowledge to select one design that is best suited to the goals of the research. Judicious selection of the 
design improves the probability that the observed change in the dependent variable was caused by 
the manipulation of the independent variable and not by another factor. It simultaneously strengthens 
the generalizability of results beyond the experimental setting. 
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Selecting and Assigning Participants 


The participants selected for the experiment should be representative of the population to which the 
researcher wishes to generalize the study’s results. This may seem self-evident, but we have witnessed 
several decades of experimentation with college sophomores that contradict that assumption. In the 
sales presentation example, corporate buyers, purchasing managers, or others in a decision-making 
capacity would provide better generalizing power than undergraduate college students if the product in 
question was targeted for industrial use rather than to the consumer. 

The procedure for random sampling of experimental subjects is similar in principle to the selec- 
tion of respondents for a survey. The researcher first prepares a sampling frame and then assigns the 
subjects for the experiment to groups using a randomization technique. Systematic sampling may be 
used if the sampling frame is free from any form of periodicity that parallels the sampling ratio. Since 
the sampling frame is often small, experimental subjects are recruited; thus, they are a self-selecting 
sample. However, if randomization is used, those assigned to the experimental group are likely to be 
similar to those assigned to the control group. Random assignment to the groups is required to make 
the groups as comparable as possible with respect to the dependent variable. Randomization does 
not guarantee that if a pretest of the groups was conducted before the treatment condition, the groups 
would be pronounced identical; but it is an assurance that those differences remaining are randomly 
distributed. In our example, we would need three randomly assigned groups—one for each of the two 
treatments and one for the control group. 

When it is not possible to randomly assign subjects to groups, matching may be used. Matching 
employs a nonprobability quota sampling approach. The object of matching is to have each experi- 
mental and control subject matched on every characteristic used in the research. This becomes more 
cumbersome as the number of variables and groups in the study increases. Since the characteristics of 
concern are only those that are correlated with the treatment condition or the dependent variable, they 
are easier to identify, control, and match.* In the sales presentation experiment, if a large part of the 
sample was composed of businesswomen who had recently completed communications training, we 
would not want the characteristics of gender, business experience, and communication training to be 
disproportionately assigned to one group. 

Some authorities suggest a quota matrix as the most efficient means of visualizing the matching 
process.° In Exhibit 9-3, one-third of the subjects from each cell of the matrix would be assigned to 
each of the three groups. If matching does not alleviate the assignment problem, a combination of 
matching, randomization, and increasing the sample size would be used. 


Pilot Testing, Revising, and Testing 


The procedures for this stage are similar to those for other forms of primary data collection. Pilot test- 
ing is intended to reveal errors in the design and improper control of extraneous or environmental con- 
ditions. Pretesting the instruments permits refinement before the final test. This is the researcher’s best 
opportunity to revise scripts, look for control problems with laboratory conditions, and scan the envi- 
ronment for factors that might confound the results. In field experiments, researchers are sometimes 
caught off guard by events that have a dramatic effect on subjects: the test marketing of a competitor’s 
product announced before an experiment, or a reduction in force, reorganization, or merger before a 
crucial organizational intervention. The experiment should be timed so that subjects are not sensitized 
to the independent variable by factors in the environment. 


Analyzing the Data 


If adequate planning and pretesting have occurred, the experimental data will take an order and struc- 
ture uncommon to surveys and unstructured observational studies. It is not that data from experiments 
are easy to analyze; they are simply more conveniently arranged because of the levels of the treatment 
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>Exhibit 9-3 Quota Matrix Example 


Category Frequencies before Matching 


Women Men 
Business No Business Business No Business 
Experience Experience Experience Experience 


Training in 


Bes : 45 
communication skills 
No training in 
communication skills 39 


Group Composition after Matching 


Experimental Groups Control 
Group 


- | OHOFOH®@: 
-|OHOHOH®': 
-|OCOHOLOEH® 


ie) 
foe) 
ye) 
(oe) 
ye) 
(oe) 
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condition, pretests and posttests, and the group structure. The choice of statistical techniques is com- 
mensurately simplified. 

Researchers have several measurement and instrument options with experiments. Among 
them are: 

¢ Observational techniques and coding schemes. 

e Paper-and-pencil tests. 

° Self-report instruments with open-ended or closed questions. 

e Scaling techniques (e.g., Likert scales, semantic differentials, Q-sort). 


e Physiological measures (e.g., galvanic skin response, EKG, voice pitch analysis, eye dilation). 
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> Validity in Experimentation 


Even when an experiment is the ideal research design, it is not without problems. There is always a 
question about whether the results are true. We have previously defined validity as whether a measure 
accomplishes its claims. Although there are several different types of validity, here only the two major 
varieties are considered: internal validity—do the conclusions we draw about a demonstrated experi- 
mental relationship truly imply cause?—and external validity—does an observed causal relationship 
generalize across persons, settings, and times?° Each type of validity has specific threats we need to 
guard against. 


Internal Validity 


Among the many threats to internal validity, we consider the following seven: 


e History 

e Maturation 

¢ Testing 

e Instrumentation 

e Selection 

¢ Statistical regression 


e Experimental mortality 


History 


During the time that an experiment is taking place, some events may occur that confuse the rela- 
tionship being studied. In many experimental designs, we take a control measurement (O,) of the 
dependent variable before introducing the manipulation (X). After the manipulation, we take an after- 
measurement (O,) of the dependent variable. Then the difference between O, and O, is the change that 
the manipulation has caused. 

A company’s management may wish to find the best way to educate its workers about the financial 
condition of the company before this year’s labor negotiations. To assess the value of such an effort, 
managers give employees a test on their knowledge of the company’s finances (O,). Then they pres- 
ent the educational campaign (X) to these employees, after which they again measure their knowledge 
level (O,). This design, known as a pre-experiment because it is not a very strong design, can be dia- 
grammed as follows: 


O x O 


ft 2, 
Pretest Manipulation _Posttest 


Between O F and O,, however, many events could occur to confound the effects of the education effort. 
A newspaper article might appear about companies with financial problems, a union meeting might be 
held at which this topic is discussed, or another occurrence could distort the effects of the company’s 
education test. 


Maturation 


Changes also may occur within the subject that are a function of the passage of time and are not specific 
to any particular event. These are of special concern when the study covers a long time, but they may 
also be factors in tests that are as short as an hour or two. A subject can become hungry, bored, or tired 
in a short time, and this condition can affect response results. 
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Testing 


The process of taking a test can affect the scores of a second test. The mere experience of taking the 
first test can have a learning effect that influences the results of the second test. 


Instrumentation 


This threat to internal validity results from changes between observations in either the measuring in- 
strument or the observer. Using different questions at each measurement is an obvious source of po- 
tential trouble, but using different observers or interviewers also threatens validity. There can even be 
an instrumentation problem if the same observer is used for all measurements. Observer experience, 
boredom, fatigue, and anticipation of results can all distort the results of separate observations. 


Selection 


An important threat to internal validity is the differential selection of subjects for experimental and 
control groups. Validity considerations require that the groups be equivalent in every respect. If sub- 
jects are randomly assigned to experimental and control groups, this selection problem can be largely 
overcome. Additionally, matching the members of the groups on key factors can enhance the equiva- 
lence of the groups. 


Statistical Regression 


This factor operates especially when groups have been selected by their extreme scores. Suppose we 
measure the output of all workers in a department for a few days before an experiment and then conduct 
the experiment with only those workers whose productivity scores are in the top 25 percent and bottom 
25 percent. No matter what is done between O, and O,, there is a strong tendency for the average of the 
high scores at O, to decline at O, and for the low scores at O, to increase. This tendency results from 
imperfect measurement that, in effect, records some persons abnormally high and abnormally low at O.. 
In the second measurement, members of both groups score more closely to their long-run mean scores. 


Experiment Mortality 


This occurs when the composition of the study groups changes during the test. Attrition is especially 
likely in the experimental group, and with each dropout the group changes. Because members of the 
control group are not affected by the testing situation, they are less likely to withdraw. In a compen- 
sation incentive study, some employees might not like the change in compensation method and may 
withdraw from the test group; this action could distort the comparison with the control group that has 
continued working under the established system, perhaps without knowing a test is under way. 


All the threats mentioned to this point are generally, but not always, dealt with adequately in experiments 
by random assignment. However, five additional threats to internal validity are independent of whether or 
not one randomizes.’ The first three have the effect of equalizing experimental and control groups. 


1. Diffusion or imitation of treatment. If people in the experimental and control groups talk, then those 
in the control group may learn of the treatment, eliminating the difference between the groups. 


2. Compensatory equalization. Where the experimental treatment is much more desirable, there 
may be an administrative reluctance to deprive the control group members. Compensatory 
actions for the control groups may confound the experiment. 


3. Compensatory rivalry. This may occur when members of the control group know they are in 
the control group. This may generate competitive pressures, causing the control group mem- 
bers to try harder. 


4. Resentful demoralization of the disadvantaged. When the treatment is desirable and the experi- 
ment is obtrusive, control group members may become resentful of their deprivation and lower 
their cooperation and output. 
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5. Local history. The regular history effect already mentioned impacts both experimental and 
control groups alike. However, when one assigns all experimental persons to one group session 
and all control people to another, there is a chance for some idiosyncratic event to confound re- 
sults. This problem can be handled by administering treatments to individuals or small groups 
that are randomly assigned to experimental or control sessions. 


External Validity 


Internal validity factors cause confusion about whether the experimental treatment (X) or extraneous 
factors are the source of observation differences. In contrast, external validity is concerned with the 
interaction of the experimental treatment with other factors and the resulting impact on the ability to 
generalize to (and across) times, settings, or persons. Among the major threats to external validity are 
the following interactive possibilities: 


¢ Reactivity of testing on X. 
e Interaction of selection and X. 


e Other reactive factors. 


The Reactivity of Testing on X 


The reactive effect refers to sensitizing subjects via a pretest so that they respond to the experimental 
stimulus (X) in a different way. A before-measurement of a subject’s knowledge about the ecology 
programs of a company will often sensitize the subject to various experimental communication efforts 
that might be made about the company. This before-measurement effect can be particularly significant 
in experiments where the IV is a change in attitude. 


Interaction of Selection and X 


The process by which test subjects are selected for an experiment may be a threat to external validity. 
The population from which one selects subjects may not be the same as the population to which one 
wishes to generalize results. Suppose you use a selected group of workers in one department for a test 
of the piecework incentive system. The question may remain as to whether you can extrapolate those 
results to all production workers. Or consider a study in which you ask a cross section of a population 
to participate in an experiment but a substantial number refuse. If you conduct the experiment only with 
those who agree to participate (self-selection), can the results be generalized to the total population? 


Other Reactive Factors 


The experimental settings themselves may have a biasing effect on a subject’s response to X. An artifi- 
cial setting can obviously produce results that are not representative of larger populations. Suppose the 
workers who are given the incentive pay are moved to a different work area to separate them from the 
control group. These new conditions alone could create a strong reactive condition. 

If subjects know they are participating in an experiment, there may be a tendency to role-play in 
a way that distorts the effects of X. Another reactive effect is the possible interaction between X and 
subject characteristics. An incentive pay proposal may be more effective with persons in one type of 
job, with a certain skill level, or with a certain personality trait. 

Problems of internal validity can be solved by the careful design of experiments, but this is less 
true for problems of external validity. External validity is largely a matter of generalization, which, in 
a logical sense, is an inductive process of extrapolating beyond the data collected. In generalizing, we 
estimate the factors that can be ignored and that will interact with the experimental variable. Assume 
that the closer two events are in time, space, and measurement, the more likely they are to follow the 
same laws. As a rule of thumb, first seek internal validity. Try to secure as much external validity as 
is compatible with the internal validity requirements by making experimental conditions as similar as 
possible to conditions under which the results will apply. 
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Wendy’s Chooses Columbus for Burger Experiment 


When a firm chooses a location to research a new product 
launch, it wants a representative sample. Some might think that 
means an “average,” place. But according to Alex Fischer of the 
Columbus Partnership, companies choose Columbus, Ohio, 
not because it is average but because it’s not. With more than 
a dozen universities (The Ohio State University included), “you 
have this interesting blend of a funky culture, creativity, diversity, 
and a corporate culture that really mix it up.” 

That’s why Columbus was a great choice for Wendy’s In- 
ternational as the perfect place to experiment with several new 
upscale sandwiches, the Black Label burgers. Black Label burg- 


has already bypassed Burger King as the second-largest fast- 
food chain behind McDonald's. 

The research that is launching the Black Label burgers in- 
cludes volunteer testers in a lab setting rating a burger prepared 
by lab technicians to precise specifications on numerous at- 
tributes, including such things as onion flavor and messiness. 
This is followed by tweaking of recipes and then an in-store test 
where actual customers taste the sandwiches and provide a 
set of performance scores. “We're still waiting on our consumer 
scores, but I’m anticipating . . . that they’re going to be great,” 
shared senior vice president Lori Estrada. 


ers are the newest sandwiches in Wendy’s strategy to capture 
business from fast casual restaurants like Five Guys. Wendy’s 


www.wendys.com 


> Experimental Research Designs 


The many experimental designs vary widely in their power to control contamination of the relationship 
between independent and dependent variables. The most widely accepted designs are based on this char- 
acteristic of control: (1) preexperiments, (2) true experiments, and (3) field experiments (see Exhibit 9-4). 


Preexperimental Designs 


All three preexperimental designs are weak in their scientific measurement power—that is, they fail to 
control adequately the various threats to internal validity. This is especially true of the after-only study. 


After-Only Study 


This may be diagrammed as follows: 


xX O 
Treatment or manipulation Observation or measurement (1) 
of independent variable of dependent variable 


An example is an employee education campaign about the company’s financial condition without a 
prior measurement of employee knowledge. Results would reveal only how much the employees know 
after the education campaign, but there is no way to judge the effectiveness of the campaign. How well 
do you think this design would meet the various threats to internal validity? The lack of a pretest and 
control group makes this design inadequate for establishing causality. 


One-Group Pretest-Posttest Design 


This is the design used earlier in the educational example. It meets the various threats to internal valid- 
ity better than the after-only study, but it is still a weak design. How well does it control for history? 
Maturation? Testing effect? The others? 


O xX O 
Pretest Manipulation — Posttest 


(2) 
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>Exhibit 9-4 Key to Design Symbols 


An X represents the introduction of an experimental 
x stimulus to a group. The effects of this independent 
variable(s) are of major interest. 


O An O identifies a measurement or observation activity. 


R An R indicates that the group members have been 
randomly assigned to a group. 


An E represents the effect of the experiment and is 
presented as an equation. 


The Xs and Os in the diagram are read from left to right in 
temporal order. 


Oxoo 


= 
Time 


When multiple Xs and Os appear vertical to each other, this indicates 
that the stimuli and/or the observations take place simultaneously. 


x 


Time 
q 


Parallel rows that are not separated by dashed lines indicate that 
comparison groups have been equalized by the randomization process. 


KO 


O 
Those separated with a dashed line have not been so equalized. 


Static Group Comparison 


This design provides for two groups, one of which receives the experimental stimulus while the other 
serves as a control. In a field setting, imagine this scenario. A forest fire or other natural disaster is 
the experimental treatment, and psychological trauma (or property loss) suffered by the residents 
is the measured outcome. A pretest before the forest fire would be possible, but not on a large scale 
(as in the California fires). Moreover, timing of the pretest would be problematic. The control group, 
receiving the posttest, would consist of residents whose property was spared. 


— 3) 


The addition of a comparison group creates a substantial improvement over the other two designs. 
Its chief weakness is that there is no way to be certain that the two groups are equivalent. 
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The Power of a Message to Change Beverage Consumption 


New food labeling requirements went into effect in 2012 that 
require more nutritional labeling to help people make better 
choices. Yet the Nielsen Global Survey of Food labeling with 
more than 25,000 respondents in 56 countries revealed that 59 
percent have trouble understanding these labels. 

One John Hopkins researcher, noting the strong link be- 
tween sugar-sweetened beverages (SSBs) and obesity in black 
teens, designed an experiment to see if a different messaging 
approach could encourage teens to avoid such beverages. 

Using four Baltimore, MD corner stores, all situated in neigh- 
borhoods with at least 70 percent black populations and located 
no more than five blocks from a middle or high school—to reach 
the targeted 12-18 year old subjects—the researcher tested 
three different messages on 8¥2 by 11-inch signs posted at 
eye level on beverage case doors. One sign offered absolute 
calories alone. Another sign offered calories as a percent of daily 
intake. Another offered the physical activity needed to burn off 
the calories. 

“Did you know that a bottle of soda or fruit juice:” 


e Has about 250 calories (treatment #1) 
e Has 10 percent of your daily calories (treatment #2) 
e Takes 50 minutes of running to work off (treatment #3) 


A researcher, located in an unobtrusive location, selected 
a random sample of subjects as they entered the store and 
then observed those subjects approach the beverage case 
and make a selection. The researcher recorded whether a 
sugary beverage or a non-sugary alternative was purchased. 
The same researcher was used in all four stores to control for 
researcher bias. 

The experiment was conducted from May through Octo- 
ber, with two stores active in May/June, two stores active in 
September/October, and all four stores active in July/August, 
to account for any differences in consumption patterns due to 
seasonality. 

The percentage of sugar-sweetened beverages were tallied 
using all beverage purchases as the denominator and compared 
with the baseline of no signage. “Absolute Calories” and “Percent 
of Daily Intake” signs reduced the likelihood of sugar-sweetened 


purchases by 40 percent, compared with the baseline of no sig- 
nage. “Physical activity equivalent” sign reduced the likelihood of 
purchase by 50 percent. 

SSBs accounted for 93.3 percent of all beverage purchases, 
compared to 87.5 percent during treatment #1, 86.5 percent 
during treatment #2, and 86.0 percent during treatment #3. Pur- 
chases of iced tea and sports drinks declined after posting of the 
signs, while purchases of water increased. 

Using experimental design symbols, plot the design of this 
experiment. 


True Experimental Designs 


The major deficiency of the preexperimental designs is that they fail to provide comparison groups 
that are truly equivalent. The way to achieve equivalence is through matching and random assign- 
ment. With randomly assigned groups, we can employ tests of statistical significance of the observed 
differences. 
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It is common to show an X for the test stimulus and a blank for the existence of a control situation. 
This is an oversimplification of what really occurs. More precisely, there is an X, and an X,, and some- 
times more. The X, identifies one specific independent variable, while X, is another independent vari- 
able that has been chosen, often arbitrarily, as the control case. Different levels of the same independent 
variable may also be used, with one level serving as the control. 


Pretest-Posttest Control Group Design 


This design consists of adding a control group to the one-group pretest—posttest design and assigning 
the subjects to either of the groups by a random procedure (R). The diagram is: 


£O, 2-0, 
R 0, O, (4) 


The effect of the experimental variable is 
B=(O, =O) =, = 0) 


In this design, the seven major internal validity problems are dealt with fairly well, although there are 
still some difficulties. Local history may occur in one group and not the other. Also, if communication 
exists between people in test and control groups, there can be rivalry and other internal validity problems. 

Maturation, testing, and regression are handled well because one would expect them to be felt 
equally in experimental and control groups. Mortality, however, can be a problem if there are different 
dropout rates in the study groups. Selection is adequately dealt with by random assignment. 

The record of this design is not as good on external validity, however. There is a chance for a reactive 
effect from testing. This might be a substantial influence in attitude change studies where pretests intro- 
duce unusual topics and content. Nor does this design ensure against reaction between selection and the 
experimental variable. Even random selection may be defeated by a high decline rate by subjects. This 
would result in using a disproportionate share of people who are essentially volunteers and who may 
not be typical of the population. If this occurs, we will need to replicate the experiment several times 
with other groups under other conditions before we can be confident of external validity. 


Posttest-Only Control Group Design 


In this design, the pretest measurements are omitted. Pretests are well established in classical research 
design but are not really necessary when it is possible to randomize. The design is: 


R X O, 
R O (5) 


z 


The experimental effect is measured by the difference between O, and O,: 
£=(2,=C)) 


The simplicity of this design makes it more attractive than the pretest—posttest control group design. 
Internal validity threats from history, maturation, selection, and statistical regression are adequately 
controlled by random assignment. Since the participants are measured only once, the threats of testing 
and instrumentation are reduced, but different mortality rates between experimental and control groups 
continue to be a potential problem. The design reduces the external validity problem of testing interac- 
tion effect. 


Field Experiments: Quasi- or Semi-Experiments"° 


Under field conditions, we often cannot control enough of the extraneous variables or the experimental 
treatment to use a true experimental design. Because the stimulus condition occurs in a natural environ- 
ment, a field experiment is required. 

A modern version of the bystander and thief field experiment, mentioned at the beginning of the 
chapter, involves the use of electronic article surveillance to prevent shrinkage due to shoplifting. In a 
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A Job Enrichment Quasi-Experiment® 


One theory of job attitudes holds that “hygiene” factors, which 
include working conditions, pay, security, status, interpersonal 
relationships, and company policy, can be a major source of 
dissatisfaction among workers but have little positive motiva- 
tional power. This theory says that the positive motivator factors 
are intrinsic to the job; they include achievement, recognition 
for achievement, the work itself, responsibility, and growth or 
advancement.® 

A study of the value of job enrichment as a builder of job 
satisfaction was carried out with laboratory technicians, or “ex- 
perimental officers” (EOs), at British Chemical. The project was a 
multiple group time series quasi-experiment. The project is dia- 
grammed at the end of this Closeup. 

Two sections of the department acted as experimental 
groups and two sections acted as control groups. It is not clear 
how these groups were chosen, but there was no mention of 
random assignment. One of the experimental groups and one 
of the control groups worked closely together, while the other 
two groups were separated geographically and were engaged 
in different research. Hygiene factors were held constant during 
the research, and the studies were kept confidential to avoid the 
tendency of participants to act in artificial ways. 

A before-measurement was made using a job reaction sur- 
vey instrument. This indicated the EOs typically had low morale, 
and many wrote of their frustrations. All EOs were asked to write 
monthly progress reports, and these were used to assess the 
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quality of their work. The assessment was made against eight 
specifically defined criteria by a panel of three managers who 
were not members of the department. These assessors were 
never told which laboratory technicians were in the experimental 
group and which were in the control group. 

The study extended over a year, with the treatments intro- 
duced in the experimental groups at the start of the 12-month 
study period. Changes were made to give experimental group 
EOs important chances for achievement; these changes also 
made the work more challenging. Recognition of achievement 
was given, authority over certain aspects was increased, new 
managerial responsibilities were assigned to the senior EOs, 
added advancements were given to others, and the opportu- 
nity for self-initiated work was provided. After about six months, 
these same changes were instituted with one of the control 
groups, while the remaining group continued for the entire 
period as a control. Several months of EO progress reports were 
available as a prior baseline for evaluation. The results of this 
project are shown in Exhibit 9-5. 
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proprietary study, a shopper came to the optical counter of an upscale mall store and asked to be shown 
special designer frames. The salesperson, a confederate of the experimenter, replied that she would get 
them from a case in the adjoining department and disappeared. The “thief” selected two pairs of sun- 
glasses from an open display, deactivated the security tags at the counter, and walked out of the store. 

Thirty-five percent of the subjects (store customers) reported the theft upon the return of the sales- 
person. Sixty-three percent reported it when the salesperson asked about the shopper. Unlike previous 
studies, the presence of a second customer did not reduce the willingness to report a theft. 

This study was not possible with a control group, a pretest, or randomization of customers; but the 
information gained was essential and justified a compromise of true experimental designs. We use the 
preexperimental designs previously discussed or quasi-experiments to deal with such conditions. In 
a quasi-experiment, we often cannot know when or to whom to expose the experimental treatment. 
Usually, however, we can decide when and whom to measure. A quasi-experiment is inferior to a true 
experimental design but is usually superior to preexperimental designs. In this section, we consider a 
few common quasi-experiments. 


Nonequivalent Control Group Design 


This is a strong and widely used quasi-experimental design. It differs from the pretest—posttest con- 
trol group design, because the test and control groups are not randomly assigned. The design is dia- 
grammed as follows: 


ey ae (6) 


There are two varieties. One is the intact equivalent design, in which the membership of the 
experimental and control groups is naturally assembled. For example, we may use different classes in 
a school, membership in similar clubs, or customers from similar stores. Ideally, the two groups are as 
alike as possible. This design is especially useful when any type of individual selection process would 
be reactive. 

The second variation, the self-selected experimental group design, is weaker because volunteers 
are recruited to form the experimental group, while nonvolunteer subjects are used for control. Such a 
design is likely when subjects believe it would be in their interest to be a subject in an experiment—-say, 
an experimental training program. 

Comparison of pretest results (O, — O,) is one indicator of the degree of equivalence between test 
and control groups. If the pretest results are significantly different, there is a real question about the 
groups’ comparability. On the other hand, if pretest observations are similar between groups, there is 
more reason to believe internal validity of the experiment is good. 


Separate Sample Pretest-Posttest Design 


This design is most applicable when we cannot know when and to whom to introduce the treatment but 
we can decide when and whom to measure. The basic design is: 


R O, (&) 
R x: @, (7) 


The bracketed treatment (X) is irrelevant to the purpose of the study but is shown to suggest that the 
experimenter cannot control the treatment. 

This is not a strong design because several threats to internal validity are not handled adequately. 
History can confound the results but can be overcome by repeating the study at other times in other 
settings. In contrast, it is considered superior to true experiments in external validity. Its strength results 
from its being a field experiment in which the samples are usually drawn from the population to which 
we wish to generalize our findings. 

We would find this design more appropriate if the population were large, if a before-measurement 
were reactive, or if there were no way to restrict the application of the treatment. Assume a company is 
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planning an intense campaign to change its employees’ attitudes toward energy conservation. It might 
draw two random samples of employees, one of which is interviewed about energy use attitudes before 
the information campaign. After the campaign the other group is interviewed. 


Group Time Series Design 


A time series design introduces repeated observations before and after the treatment and allows sub- 
jects to act as their own controls. The single treatment group design has before-after measurements as 
the only controls. There is also a multiple design with two or more comparison groups as well as the 
repeated measurements in each treatment group. 

The time series format is especially useful where regularly kept records are a natural part of the 
environment and are unlikely to be reactive. The time series approach is also a good way to study 
unplanned events in an ex post facto manner. If the federal government were to suddenly begin price 
controls, we could still study the effects of this action later if we had regularly collected records for the 
period before and after the advent of price control. 

The internal validity problem for this design is history. To reduce this risk, we keep a record of pos- 
sible extraneous factors during the experiment and attempt to adjust the results to reflect their influence. 


Ce 


1 Experiments are studies involving intervention by the re- 
searcher beyond that required for measurement. The usual 
intervention is to manipulate a variable (the independent 
variable) and observe how it affects the subjects being stud- 
ied (the dependent variable). 

An evaluation of the experimental method reveals several 
advantages: (1) the ability to uncover causal relationships, 
(2) provisions for controlling extraneous and environmental 
variables, (3) convenience and low cost of creating test 
situations rather than searching for their appearance in busi- 
ness situations, (4) the ability to replicate findings and thus 
rule out idiosyncratic or isolated results, and (5) the ability to 


4 We judge various types of experimental research designs by 
how well they meet the tests of internal and external validity. 
An experiment has high internal validity if one has confidence 
that the experimental treatment has been the source of 
change in the dependent variable. More specifically, a de- 
sign’s internal validity is judged by how well it meets seven 
threats. These are history, maturation, testing, instrumenta- 
tion, selection, statistical regression, and experiment mortality. 

External validity is high when the results of an experi- 
ment are judged to apply to some larger population. Such 
an experiment is said to have high external validity regarding 
that population. Three potential threats to external validity 


exploit naturally occurring events. 


Some advantages of other methods that are liabilities for the 
experiment include (1) the artificial setting of the laboratory, 
(2) generalizability from nonprobability samples, (3) dispro- 
portionate costs in select business situations, (4) a focus 
restricted to the present and immediate future, and (5) ethi- 
cal issues related to the manipulation and control of human 
subjects. 

Consideration of the following activities is essential for the 
execution of a well-planned experiment: 

Select relevant variables for testing. 

Specify the treatment levels. 

Control the environmental and extraneous factors. 
Choose an experimental design suited to the hypothesis. 


Select and assign subjects to groups. 
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Pilot test, revise, and conduct the final test. 
Analyze the data. 


Ko} 


are testing reactivity, selection interaction, and other reactive 
factors. 


5 Experimental research designs include (1) preexperiments, 
(2) true experiments, and (38) quasi-experiments. The main 
distinction among these types is the degree of control that 

he researcher can exercise over validity problems. Three 

preexperimental designs were presented in the chapter. 

These designs represent the crudest form of experimen- 

ation and are undertaken only when nothing stronger is 

possible. Their weakness is the lack of an equivalent com- 
parison group; as a result, they fail to meet many internal va- 
idity criteria. They are the (1) after-only study, (2) one-group 
pretest-posttest design, and (3) static group comparison. 
Two forms of the true experiment were also presented. 

Their central characteristic is that they provide a means by 

which we can ensure equivalence between experimental 

and control groups through random assignment to the 
groups. These designs are (1) pretest-posttest control 
group and (2) posttest-only control group. 


The classical two-group experiment can be extended to 
multigroup designs in which different levels of the test vari- 
able are used as controls rather than the classical nontest 
control. 

Between the extremes of preexperiments, with little or 
no control, and true experiments, with random assignment, 
there is a gray area in which we find quasi-experiments. 
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These are useful designs when some variables can be con- 
trolled, but equivalent experimental and control groups usu- 
ally cannot be established by random assignment. There are 
many quasi-experimental designs, but only three were cov- 
ered in this chapter: (1) nonequivalent control group design, 
(2) separate sample pretest-posttest design, and (3) group 
time series design. 
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Terms in Review 
1 Distinguish between the following: 


a_ Internal validity and external validity. 
Preexperimental design and quasi-experimental design. 
History and maturation. 


Random sampling, randomization, and matching. 


oaan4», 


Environmental variables and extraneous variables. 


2 Compare the advantages of experiments with the advan- 
tages of survey and observational methods. 


3 Why would a noted business researcher say, “It is essential 
that we always keep in mind the model of the controlled 
experiment, even if in practice we have to deviate from an 
ideal model’? 


4 What ethical problems do you see in conducting experi- 
ments with human subjects? 


5 What essential characteristics distinguish a true experiment 
from other research designs? 


Making Research Decisions 
6 A lighting company seeks to study the percentage of de- 
fective glass shells being manufactured. Theoretically, the 
percentage of defectives is dependent on temperature, 
humidity, and the level of artisan expertise. Complete 


historical data are available for the following variables on a 
daily basis for a year: 


a Temperature (high, normal, low). 
b Humidity (high, normal, low). 
c Artisan expertise level (expert, average, mediocre). 


Some experts feel that defectives also depend on produc- 
tion supervisors. However, data on supervisors in charge 
are available for only 242 of the 365 days. How should this 
study be conducted? 


Much Internet advertising is priced based on click-through 
activity. A prospect is shown an ad on a host website 
based on search words he or she might have entered in 

a search engine such as Google or Bing. If the prospect 
clicks directly on the ad they see on the host website to 
visit the advertiser’s site, the ad is considered effective and 
the advertiser must pay the host website for the ad. But 
research reveals that while all prospects do not click on 
the ad they are shown on a host website, many do visit the 
advertiser’s site. They simply key in the advertiser’s URL 
directly into their browser or search engine. How would 
you design an experiment to determine if non-click-through 
ads displayed on your host website were actually effective 
in getting a prospect to an advertiser's website? 
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8 A pharmaceuticals manufacturer is testing a drug developed 
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to treat cancer. During the final stages of development the 
drug’s effectiveness is being tested on individuals for differ- 
ent (1) dosage conditions and (2) age groups. One of the 
problems is patient mortality during experimentation. Justify 
your design recommendations through a comparison of 
alternatives and in terms of external and internal validity. 


a Recommend the appropriate design for the experiment. 


b Explain the use of control groups, blinds, and double 
blinds if you recommend them. 


You are asked to develop an experiment for a study of the 
effect that compensation has on the response rates secured 
from personal interview subjects. This study will involve 300 
people who will be assigned to one of the following condi- 
tions: (1) no compensation, (2) $1 compensation, and (3) 

$3 compensation. A number of sensitive issues will be 
explored concerning various social problems, and the 300 
people will be drawn from the adult population. Describe your 
design. You may find Appendix 9a valuable for this question. 


What type of experimental design would you recommend 
in each of the following cases? Suggest in some detail how 
you would design each study: 


a A test of three methods of compensation of factory 
workers. The methods are hourly wage, incentive pay, 
and weekly salary. The dependent variable is direct 
labor cost per unit of output. 


b Astudy of the effects of various levels of advertising 
effort and price reduction on the sale of specific 
branded grocery products by a retail grocery chain. 


c Astudy to determine whether it is true that the use of 
fast-paced music played over a store’s public address 
system will speed the shopping rate of customers with- 
out an adverse effect on the amount spent per customer. 


Bringing Research to Life 
11 Design an experiment for the opening vignette. 


From Concept to Practice 

12 Using Exhibit 9-4, diagram an experiment described in one 
of the Snapshots in this chapter using research design 
symbols. 


From the Headlines 

13 One of the hardest aspects of a merger is making the IT sys- 
tems of the merging companies talk with each other. That 
problem grows in magnitude when you are talking about 
merging airlines, Continental and United Airlines. While the 
ticketing conversion worked well, the distinctness of the air 
traffic tracking systems made the conversion problematic. 
So when the company decided that United’s system was 
more suitable for the merged airline, it needed to test it. 
How would you design an experiment to see if your track- 
ing system could know where flights were, what their arrival 
times and departure times were, what the flight numbers 
were, and whether they deviated from their flight plan? 


er 


McDonald’s Tests Catfish Sandwich 


NetConversions Influences Kelley Blue Book 


*You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. 


>online learnin nter 


You'll find the following appendices available from the Online Learning Center to supplement the content of this chapter: 


Appendix 9a: Complex Experimental Designs 
Appendix 9b: Test Markets 
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learningobjectives 
After reading this chapter, you should understand... 


The process for selecting the appropriate and optimal communication approach. 
What factors affect participation in communication studies. 

The major sources of error in communication studies and how to minimize them. 
The major advantages and disadvantages of the three communication approaches. 


Why an organization might outsource a communication study. 


6 6 If social mediais the temperature check, surveys 
are the taste test to validate that the meal is 
cooked properly. A good survey enables you to: 
(1) tune out the noise and zero in on the opinions 
and comments that are directly applicable to 


your most pressing questions, and (2) validate 
trends and shifts in preference that you may 
have suspected, but weren’t certain about. 9 9 


Reggie Aggarwal, CEO, 
Cvent 


“Jason, you'll enjoy this one,” Sara comments as 
she joins him for their meeting to discuss the Albany 
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Outpatient Laser Clinic patient satisfaction project. She nods as she passes it across the desk. 


extends the letter and smiles widely. 


Edna Koogan, P. A., Attorney at Law 
P. O. Box 8219-2767 
Albany, New York 12212-2767 


Dr. Edith Coblenz, M.D. 
3456 Barshoot Building 
Albany, New York 12212 


Dear Edith, 

I want you to have my side of this morning’s incident at the Albany Outpatient Laser Clinic 
Inc. I am sure you have by now heard from the business manager and the admissions director 
and possibly the anesthetist. You are a stockholder in the center, I know, and as your former 
lawyer and current patient, I thought I owed you a warning and explanation. 

You told me to report to the center at 7 a.m. for a workup in preparation for eye surgery to- 
morrow. I caught a cab and was there at 6:55 promptly. I identified myself as your patient, and 
at once the receptionist called someone from the back room and said, “Ms. Koogan’s personal 
physician is Dr. Coblenz,” which is, of course, not true, as you are my eye doctor. But I was 
too cold to argue since they had left us standing in the snow until 7:10. 

A fellow insisted on taking my glasses and medications with him “for a workup.” As 
soon as he disappeared with my glasses a second admissions clerk appeared and handed me a 
“questionnaire” to fill out. It appeared to be a photocopy of a photocopy of a photocopy and 
was very faintly printed in small gray type on a light gray sheet. When I pointed out that I was 
about to be admitted for treatment of glaucoma, a leading cause of blindness, she told me, “Do 
the best you can.” When I objected emphatically, she seemed taken aback. I suppose most of 
her 80-year-old patients are more compliant, but I guess I am an intractable old attorney. 

Was I wrong to object to the questionnaire being too faint and the type too small? Am I 
the first glaucoma patient who has ever been treated at the Laser Center? One would think 
they would understand you can’t ask someone blind in one eye to fill out such a questionnaire, 
especially without her glasses. The clerk finally, grudgingly, asked me to sit by her side, so she 
could help me. 

There were several questions about my name, address, age, and occupation. Then she 
wanted to know the name of the admitting physician and then the phone number (but not the 
name) of the physician who was most familiar with my health. I said the admitting physician 
was an eye doctor and the physician most familiar with my health was a GP, and asked, which 
did she want the phone number for, the eye doctor or the GP? She admonished me to try and 
“get over that bad attitude.” Then she told me to go fill out the form as best I could. 


“Is that the letter that clinic administrator George 
Bowlus promised he’d send over this morning?” Sara 
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A very nice patient (hemorrhoids, no vision problems) offered to help me. She began read- 
ing the questionnaire and came to the item “Past Medical History: Yes or No.” She didn’t think 
this made any sense, and neither did I, because everyone has a past medical history, and no one 
would answer no; but after a while we decided that it meant I should answer yes or no to all 
of the questions underneath, such as: Did I have diabetes? Did I have heart disease? When we 
came to “Have you ever had or been treated for the flu?” we could not decide if it meant, have 
Tever had the flu? Or have I had flu recently (I had flu six months ago, but is that “‘recent’’?) so 
we asked the receptionist. She became almost speechless and said she would get me some help. 

After a while the “help” appeared—a nurse who wanted to measure my blood pressure 
and induce me to take a blue pill, which she said would be good for my “nerves.” I refused 
and pointed out rather curtly that this was not a gulag but an admissions department, a place 
of business, for crying out loud, where they ought to be able to handle a little criticism from 
someone trained to elicit accurate information. 

By then several nice people had pitched in to help me with the questionnaire. But this made 
it even harder to decide on the answers, because we understood so many of the questions dif- 
ferently and couldn’t agree. When we came to “Are all your teeth intact?” One man thought 
it meant, “Do you have false teeth?” And another thought it meant, “Do you have any broken 
dentures?” But a woman who assured me her son is a dentist said it meant, “Do you have any 
loose teeth?” We couldn’t decide how to settle this issue. 

Then there was the question “Do you have limited motion of your neck?” and by then 
everyone was enjoying the incongruity of these questions. Of course I have limited motion of 
the neck. Doesn’t everyone? We decided to save that question for later clarification. 

After all of the yes-no questions there came various other stumpers, such as “Please list 
your current medications.” The problem is, of course, that I have purple eye drops and yel- 
low eye drops, but the young man had taken them away from me “for a workup,” so I had no 
way of accurately answering the questions. I was pretty sure one of them was glucagon, so I 
guessed and put that down, but then I had second thoughts and scratched it out. (When I got 
home, I checked and it was betagan, not glucagon.) 

There were four of us working on the questionnaire by then, and we were laughing and 
crowing and having a high time and discharging our anxieties, which further annoyed the admis- 
sions clerk. So she called the anesthetist, a stuck-up young fellow who said he had written the 
questionnaire himself and had never had any problems with it. That is when I told him, if he had 
not had any problems with this questionnaire, this proved it was better to be lucky than smart. 

He said he was going to overlook my “attitude” because he knew I was old and anxious 
about the coming operation. I told him I was going to take my business somewhere else be- 
cause of the bilaterality problem. ““What is that?” he asked. I said, I have two eyes, and if any- 
one as dumb as him went after me with a laser, he would probably cut the wrong eye. 

I caught a cab and sent my neighbor back for my glasses. As your lawyer, I urge you not to 
further involve yourself with such fools. 


Edna 


“Tt would seem that Albany Clinic might need help with questionnaire development,” comments Jason. Sara 


responds sarcastically, “You think?” 
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Research designs can be classified by the approach used to gather primary data. There are two alterna- 
tives. We can observe conditions, behavior, events, people, or processes. Or we can communicate with 
people about various topics, including participants’ attitudes, motivations, intentions, and expectations. 
The researcher determines the appropriate data collection approach largely by identifying the types 
of information needed—investigative questions the researcher must answer. As researchers we learn 
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>Exhibit 10-1 Data Collection Approach: Impact on the Research Process 


Select 
Data 
Collection 
Approach 


No interviewer 
involved 


No contact 


Contact Participant 
Chosen as Sample Element 


Initiate Follow-up 
Procedure 


Initiate Callback 
Procedure 


Contact made 


Present Survey to Participant 


Collect & Process data 


3s. Part B ee | —— Part A —— 


No return of survey 


much about opinions and attitudes by communication-based research; observation techniques are inca- 
pable of revealing such critical elements. This is also true of intentions, expectations, motivations, and 
knowledge. Information about past events is often available only through surveying or interviewing 
people who remember the events. Thus, the choice of a communication versus an observation approach 
may seem an obvious one, given the directions in which investigative questions may lead. The charac- 
teristics of the sample unit—specifically, whether a participant can articulate his or her ideas, thoughts, 
and experiences—also play a role in the decision. Part A of Exhibit 10-1 shows the relationship of these 
decisions to the research process detailed in Chapter 4. Part B indicates how the researcher’s choice of 
a communication approach affects the following: 


e The creation and selection of the measurement questions (to be explored in Chapters 11 and 12). 


¢ Instrument design (to be discussed in Chapter 13), which incorporates attempts to reduce error 
and create participant-screening procedures. 


217 


218 >part Il The Design of Business Research 


>Exhibit 10-2 Selecting a Communication Data Collection Method 


fancied Conditions, behaviors, Attitudes, motivations, Communication 
jee haw 8) events, processes intentions, expectations Approach 
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Computer 
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administered 
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Computer- 
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Interviewer-controlled 
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Intercept 


Participant-controlled 
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Interviewer Involvement 


Intercept 


Survey distribution 
only or none 


e Sampling issues (explored in Chapter 14), which drive contact and callback procedures. 


° Data collection processes, which create the need for follow-up procedures (when self- 
administered instruments are used) and possible interviewer training (when personal or 
telephone surveying methods are used). 


In this chapter we focus on the choices the researcher must make once the communication approach 
has been chosen (Exhibit 10-2). We discuss the characteristics and applications of the various commu- 
nication approaches as well as their individual strengths and weaknesses (summarized in Exhibit 10-5). 

The communication approach involves surveying or interviewing people and recording their re- 
sponses for analysis. A survey is a measurement process used to collect information during a highly 
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structured interview—sometimes with a human interviewer and other times without. Questions are 
carefully chosen or crafted, sequenced, and precisely asked of each participant. The goal of the survey 
is to derive comparable data across subsets of the chosen sample so that similarities and differences 
can be found. When combined with statistical probability sampling for selecting participants, survey 
findings and conclusions are projectable to large and diverse populations. 

The great strength of the survey as a primary data collecting approach is its versatility. Abstract in- 
formation of all types can be gathered by questioning others. Additionally, a few well-chosen questions 
can yield information that would take much more time and effort to gather by observation. A survey 
that uses the telephone, mail, a computer, e-mail, or the Internet as the medium of communication can 
expand geographic coverage at a fraction of the cost and time required by observation. 

The bad news for communication research is that all communication research has some error. 
Understanding the various sources of error helps researchers avoid or diminish such error. 


Error in Communication Research 


As depicted in Exhibit 10-3, there are three major sources of error in communication research: mea- 
surement questions and survey instruments, interviewers, and participants. Researchers cannot help a 
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Interview Refusal of 
Environment Participation 


(nonresponse error) 


Data or Survey 
Falsification 


Influencing 
Behaviors 


Physical 
Presence Bias 
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business decision maker answer a research question if they (1) select or craft inappropriate questions, 
(2) ask them in an inappropriate order, or (3) use inappropriate transitions and instructions to elicit 
information. We will spend considerable time in Chapters 11, 12, and 13 discovering ways to avoid 
these sources of error. 


Interviewer Error 


From the introduction to the conclusion of the interview, there are many points where the interviewer’ s 
control of the process can affect the quality of the data. Interviewer error, a major source of sampling 
error and response bias, is caused by numerous actions: 


¢ Failure to secure full participant cooperation (sampling error). The sample is likely to 
be biased if interviewers do not do a good job of enlisting participant cooperation. While 
instrument error was evident in the Albany Clinic study, there is also a question of whether 
the distributor of the survey (the receptionist) contributed to the lack of data quality in the data 
collected from Edna. Toward the end of the communication, there is some doubt about the 
seriousness with which questions were answered. Stressing the importance of the information 
for the upcoming surgery and having a receptionist trained to serve as question interpreter/ 
prober could reduce this type of error. 


¢ Failure to record answers accurately and completely (data entry error). Error may result from 
an interview recording procedure that forces the interviewer to summarize or interpret partici- 
pant answers or that provides insufficient space to record verbatim answers as provided by the 
participant. 


¢ Failure to consistently execute interview procedures. The precision of survey estimates will 
be reduced and there will be more error around estimates to the extent that interviewers are 
inconsistent in ways that influence the data. In the Albany Clinic study, providing different 
definitions (of diseases) to different clinic patients completing the medical history would 
create bias. 


¢ Failure to establish appropriate interview environment. Answers may be systematically inac- 
curate or biased when interviewers fail to appropriately train and motivate participants or fail 
to establish a suitable interpersonal setting.’ Since the Albany Clinic study asked for factual 
rather than attitudinal data, interviewer-injected bias would have been limited. If the clinic had 
required the admissions clerk (who insulted Edna by referring to her negative attitude) to also 
conduct a postsurgery interview on patient satisfaction, the results of the latter study may have 
been influenced by interviewer bias. 


° Falsification of individual answers or whole interviews. Perhaps the most insidious form of 
interviewer error is cheating. Surveying is difficult work, often done by part-time employees, 
usually with only limited training and under little direct supervision. At times, falsification of an 
answer to an overlooked question is perceived as an easy solution to counterbalance the incom- 
plete data. This easy, seemingly harmless first step can be followed by more pervasive forgery. 
It is not known how much of this occurs, but it should be of constant concern to research direc- 
tors as they develop their data collection design and to those organizations that outsource survey 
projects. 


e Inappropriate influencing behavior. It is also obvious that an interviewer can distort the results 
of any survey by inappropriate suggestions, directions, or verbal probes; by word emphasis 
and question rephrasing; by tone of voice; or by body language, facial reaction to an answer, or 
other nonverbal signals. These activities, whether intentional or merely due to carelessness, are 
widespread. This problem was investigated using a simple questionnaire and participants who 
then reported on the interviewers. The conclusion was “The high frequency of deviations from 
instructed behavior is alarming.”* 


¢ Physical presence bias. Interviewers can influence participants in unperceived subtle ways. 
Older interviewers are often seen as authority figures by young participants, who modify their 
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responses accordingly. Some research indicates that perceived social distance between inter- 
viewer and participant has a distorting effect, although the studies do not fully agree on just 
what this relationship is. 


In light of the numerous studies on the various aspects of interview bias, the safest course for research- 
ers is to recognize the constant potential for response error. 


Participant Error 


Three broad conditions must be met by participants to have a successful survey: 


e The participant must possess the information being targeted by the investigative questions. 


¢ The participant must understand his or her role in the interview as the provider of accurate 
information. 


e The participant must have adequate motivation to cooperate. 


Thus, participants cause error in two ways: whether they respond (willingness) and how they 
respond. 


Participation-Based Errors Three factors influence participation:* 


e The participant must believe that the experience will be pleasant and satisfying. 


e The participant must believe that answering the survey is an important and worthwhile use of his 
or her time. 


¢ The participant must dismiss any mental reservations that he or she might have about 
participation. 


Whether the experience will be pleasant and satisfying depends heavily on the interviewer in personal 
and telephone surveys. Typically, participants will cooperate with an interviewer whose behavior re- 
veals confidence and who engages people on a personal level. Effective interviewers are differentiated 
not by demographic characteristics but by these interpersonal skills. By confidence, we mean that most 
participants are immediately convinced they will want to participate in the study and cooperate fully 
with the interviewer. An engaging personal style is one in which the interviewer instantly establishes 
credibility by adapting to the individual needs of the participant. For the survey that does not employ 
human interpersonal influence, convincing the participant that the experience will be enjoyable is the 
task of a prior notification device or the study’s written introduction. 

For the participant to think that answering the survey is important and worthwhile, some expla- 
nation of the study’s purpose is necessary, although the amount of disclosure will vary based on 
the sponsor’s objectives. In personal or phone surveys the researcher will provide the interviewer 
with instructions for discovering what explanation is needed and supplying it. Usually, the inter- 
viewer states the purpose of the study, tells how the information will be used, and suggests what 
is expected of the participant. Participants should feel that their cooperation will be meaningful to 
themselves and to the survey results. When this is achieved, more participants will express their 
views willingly. 

As depicted in Exhibit 10-4, the quality and quantity of information secured depend heavily on 
the ability and willingness of participants to cooperate. Potential participants often have reservations 
about being interviewed that must be overcome. They may suspect the interviewer has an illegitimate 
purpose. They may view the topic as too sensitive and thus the interview as potentially embarrass- 
ing or intrusive. Or they may feel inadequate or fear the questioning will belittle them. Previous 
encounters with businesses that have attempted to disguise their sales pitch or fund-raising activities 
as a research survey can also erode participants’ willingness to cooperate. In personal and phone 
interviews, participants often react more to their feelings about the interviewer than to the content of 
the questions. 

At the core of a survey or interview is an interaction between two people or between a person and 
a questionnaire. In the interaction the participant is asked to provide information. While he or she has 


221 


222 


>part Il The Design of Business Research 


>Exhibit 10-4 Factors Influencing Participant Motivation 


Direction of Influence on Motivation 


Increase ———————> x Decrease 


~ == 


Participant Motivation 


Prestige of 
research sponsor Loneliness 


or agency Self-image as 


Liking or dutiful citizen 


Perceived compassion for Confirmation of 
importance interviewer self-importance 
of the topic 


Source: Influenced by Robert L. Kahn and Charles F. Cannell, “Interviewing,” in David L. Sills, ed., International Encyclopedia of the Social 
Sciences, vol. 8, p. 153. Copyright © 1968 by Crowell Collier and Macmillan, Inc. 


hope of some minimal personal reward—in the form of compensation for participation or enhanced 
status or knowledge—he or she has little hope of receiving any immediate or direct benefit from the 
data extracted. Thus, participant motivation is a responsibility of the researcher and the interviewer. 
Studies of reactions to many surveys show that participants can be motivated to participate in personal 
and phone interviews and, in fact, can even enjoy the experience. In one study, more than 90 percent 
of participants said the interview experience was interesting, and three-fourths reported they were will- 
ing to be interviewed again.° In intercept/self-administered studies, the interviewer’s primary role is to 
encourage participation as the participant completes the questionnaire on his or her own. Taking away 
Edna’s glasses, along with the natural anxiety associated with eye surgery, would not have encouraged 
Edna’s participation. However, the “required” nature of the information (we assume surgery would not 
commence without prior completion of the questionnaire) guaranteed Edna’s participation, no matter 
how grudgingly given. 

By failing to respond or refusing to respond, participants create a nonrepresentative sample for the 
study overall or for a particular item or question in the study. In surveys, nonresponse error occurs 
when the responses of participants differ in some systematic way from the responses of nonpartici- 
pants. This occurs when the researcher (1) cannot locate the person (the predesignated sample element) 
to be studied or (2) is unsuccessful in encouraging that person to participate. This is an especially diffi- 
cult problem when you are using a probability sample of subjects. Many studies have shown that better- 
educated individuals and those more interested in the topic participate in surveys. A high percentage of 
those who reply to a given survey have usually replied to others, while a large share of those who do 
not respond are habitual nonparticipants.® 

Researchers are not without actions to avoid or diminish the error discussed above. We will explore 
these options in detail in Chapters 11 through 14. Despite its challenges, communicating with research 
participants—and the use of the survey—is the principal method of marketing research. 


Response-Based Errors Response error is generated in two ways: when the participant fails 
to give a correct answer or fails to give the complete answer. The interviewer can do little about the 


Gamification in Research 


>chapter 10 Surveys 223 


According to the Gartner Group, 50 percent of all innovation and 
70 percent of all Global 2000 companies will be gamifying pro- 
cesses by 2015. Additionally, M2 Research, a media and enter- 
tainment research firm, estimates that spending on gamification 
projects will increase to as much as $2.8 billion by 2016. For 
those of you in Generation G, where gaming has been a primary 
form of entertainment, this might not be surprising. Gamification, 
however, is just starting to be recognized as a powerful force 
outside the entertainment industry. 

Gamification, according to Gabe Zichermann, author of 
Gamification, the blog, and Gamification in Marketing and Gami- 
fication by Design, is the process of “using game thinking and 
game mechanics to engage audiences and solve problems.” 
Gartner Group analyst Brian Burke indicates that gamification is 
destined to be used in such “non-game environments such as 
innovation, marketing, training, employee performance, health 
and social change.” 

Many survey participants consider the survey process bor- 
ing. Some of the fundamental tenets of gamification, shared 


a term for its gamified research respondents: playspondents. 
“AS our surveys become games, the people taking part aren’t 
quite respondents but they’re not quite game players either.” 
RTG’s fundamental research environment is the Playspondent 
Playhouse. Here a client can “rent a room to carry out [its] proj- 
ect, so when Avatars are made, they can be placed in a room 
with [the client’s] questions, product images and quotas and [be] 
ready to go!” 

The U.K.’s Department for Work and Pensions created an 
innovation game called Idea Street to generate ideas from its 
120,000 employees. Within the first 18 months, Idea Street had 
approximately 4,500 users who had generated 1,400 ideas. The 
collaboration element of the game generated 63 ideas that were 
implemented. 

Neurological research supports gaming as a research tech- 
nique as dopamine is released in the brain during gaming, and this 
pleasurable sensation serves as a reward for participation. The 
game world has conditioned players to seek its clearly articulated 
rewards. Many think research needs to be involved in this trend. 


Betty Adamou, CEO and founder of Research Through Gaming 
Ltd (RTG) and also editor-in-chief of Game Access, a dedicated 
gamification blog, are engagement and rewards. RTG created 


www.researchthroughgaming.com; www.gartner.com; 
www.m2research.com 


participant’s information level. Screening questions qualify participants when there is doubt about 
their ability to answer. The most appropriate applications for communication research are those where 
participants are uniquely qualified to provide the desired information. Questions can be used to inquire 
about characteristics of a participant, such as his or her household income, age, sexual preference, 
ethnicity, or family lifecycle stage. Questions can also be asked that reveal information exclusively 
internal to the participant. We include here items such as the participant’s lifestyle, attitudes, opinions, 
expectations, knowledge, motivations, and intentions. 

If we ask participants to report on events that they have not personally experienced, we need to as- 
sess the replies carefully. If our purpose is to learn what the participant understands to be the case, it 
is legitimate to accept the answers given. But if our intent is to learn what the event or situation actu- 
ally was, we must recognize that the participant is reporting secondhand data and the accuracy of the 
information declines. 

In the study of MindWriter’s CompleteCare program, only those individuals who have expe- 
rienced difficulty with their laptops and gone through the program have direct knowledge of the 
service process. Although some associates and family members are likely to have some second- 
hand knowledge of the experience, no one but the actual laptop owners is likely to give as clear a 
picture of what works and what doesn’t with CompleteCare. The laser patient, Edna, on the other 
hand, had a totally different experience when she went for surgery to correct her vision. Answers 
to many questions on the patient survey might have been known by a caregiver, especially since 
Edna was experiencing eye problems serious enough to warrant surgery. And the clinic’s admis- 
sions department could have been confident that such information was as accurate as it would have 
been if given by Edna herself. Since inaccuracy is a correctable source of error, a family or group 
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member should not be asked about another member’s experience unless there is no other way to 
get the information directly. We should not depend on secondhand sources if a more direct source 
can be found. 

Participants also cause error by responding in such a way as to unconsciously or consciously mis- 
represent their actual behavior, attitudes, preferences, motivations, or intentions (response bias). Par- 
ticipants create response bias when they modify their responses to be socially acceptable or to save 
face or reputation with the interviewer (social desirability bias), and sometimes even in an attempt to 
appear rational and logical. 

One major cause of response bias is acquiescence—the tendency to be agreeable. On the partici- 
pant’s part, acquiescence may be a result of lower cognitive skills or knowledge related to a concept 
or construct, language difficulties, or perceived level of anonymity. However, researchers can con- 
tribute to acquiescence by the speed with which they ask questions (the faster questions are asked, the 
more acquiescence) and the placement of questions in an interview (the later the question, the more 
acquiescence.)’ 

Sometimes participants may not have an opinion on the topic of concern. Under this circum- 
stance, their proper response should be “don’t know” or “have no opinion.” Some research suggests 
that most participants who chose the don’t-know response option actually possess the knowledge 
or opinion that the researcher seeks.* Participants may choose the option because they may want to 
shorten the time spent in the participation process, may be ambivalent or have conflicting opinions 
on the topic, may feel they have insufficient information to form a judgment—even though they 
actually have taken a position—don’t believe that the response choices match their position, or don’t 
possess the cognitive skills to understand the response options. If they choose the don’t-know op- 
tion for any of these reasons, studies suggest that probing for their true position will increase both 
reliability and validity of the data. However, forcing a participant to express some opinion he or she 
does not hold by withholding a don’t-know option makes it difficult for researchers to know the reli- 
ability of the answers. 

Participants may also interpret a question or concept differently from what was intended by the 
researcher. This occurs when the researcher uses words that are unfamiliar to the participant. Thus, the 
individual answers a question that is different from the one the researcher intended to ask. This problem 
is reflected in Edna’s letter concerning the clinic’s survey. 

Regardless of the reasons, each source of participant-initiated error diminishes the value of the data 
collected. It is difficult for a researcher to identify such occasions. Thus, communicated responses 
should be accepted for what they are—statements by individuals that reflect varying degrees of truth 
and accuracy. 


Choosing a Communication Method 


Once the sponsor or researcher has determined that surveying or interviewing is the appropriate data- 
collection approach, various means may be used to secure information from individuals. A researcher 
can conduct a semistructured interview or survey by personal interview or telephone or can distribute 
a self-administered survey by mail, fax, computer, e-mail, the Internet, or a combination of these. As 
noted in Exhibit 10-5, although there are commonalities among these approaches, several consider- 
ations are unique to each. 

In the last two decades of the 20th century and the first decade of the 21st century, a revolution— 
albeit a quiet one—was under way in survey research. Driven by changing technology and the need to 
make research more responsible to the bottom line and ROI objectives, the paper-and-pencil survey 
standard of the prior 60 years was replaced by a new computerized standard. Whether it goes by the 
name of “computer-assisted data collection” (CADAC), “computer-assisted survey information col- 
lection” (CASIC), or “computer-assisted interviewing” (CAD), the trend is growing. Although less 
obvious in the public sector (the U.S. government is the largest survey researcher in the world, and 
paper-and-pencil approaches still hold prominence there), in the private sector of survey research with 
households and organizations, the computer’s influence on this methodology is far-reaching. It influ- 
ences all the various data collection practices. 
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Exhibit 10-5 Comparison of Communication Approaches 


Description 


Advantages 


Disadvantages 


Self-Administered Survey 


Questionnaires are: 


a. Mailed, faxed, or couriered to be self- 
administered—with return mechanism 
generally included (denoted below as a). 


b. Computer-delivered via intranet, 
Internet, and online services —computer 
stores/forwards completed instruments 
automatically (denoted below as b). 


c. People are intercepted in a central loca- 
tion and studied via paper or computerized 


instrument—without interviewer assistance: 


e.g., restaurant and hotel comment cards 
(denoted below as c). 


e Allows contact with otherwise inaccessible 
participants (e.g., CEOs). 


e Incentives may be used to increase 
response rate. 


e Often lowest-cost option. 


e Expanded geographic coverage without 
increase in costs (a). 


e Requires minimal staff (a). 
e Perceived as more anonymous (a). 


e Allows participants time to think about 
questions (a). 


e More complex instruments can be used (b). 
e Fast access to the computer-literate (b). 
e Rapid data collection (b, c). 


e Participant who cannot be reached by 
phone (voice) may be accessible (b, c). 


e Sample frame lists viable locations rather 
than prospective participants (b, c). 


e Visuals may be used (b, c). 


e Low response rate in some modes. 


e No interviewer intervention available for 
probing or explanation (a). 


e Cannot be long or complex (a). 
e Accurate mailing lists needed (a). 


e Often participants returning survey repre- 
sent extremes of the population—skewed 
responses (a). 


e Anxiety among some participants (b). 


e Directions/software instruction needed for 
progression through the instrument (b). 


e Computer security (b). 


e Need for low-distraction environment for 
survey completion (c). 


Telephone Survey 


People selected to be part of 
the sample are interviewed 
on the telephone by a trained 
interviewer. 


e Lower costs than personal 
interview. 


Expanded geographic cover- 
age without dramatic increase 
in costs. 


Uses fewer, more highly 
skilled interviewers. 


Reduced interviewer bias. 


Fastest completion time. 


Better access to hard-to- 
reach participants through 
repeated callbacks. 


e Can use computerized 
random dialing. 


e CATI—computer-assisted 
telephone interviewing: 
Responses can be entered 
directly into a computer file to 
reduce error and cost. 


Response rate is lower than 
for personal interview. 


Higher costs if interviewing 
geographically dispersed 
sample. 


Interview length must be 
limited. 


e Many phone numbers are un- 
listed or not working, making 
directory listings unreliable. 


e Some target groups are not 
available by phone. 


Responses may be less 
complete. 


lllustrations cannot be used. 


Survey via Personal Interview 


People selected to be part of 
the sample are interviewed in 
person by a trained interviewer. 


e Good cooperation from 
participants. 


e Interviewer can answer 
questions about survey, 
probe for answers, use 
follow-up questions, and 
gather information by 
observation. 


e Special visual aids and 
scoring devices can be used. 


e literate and functionally 
illiterate participants can be 
reached. 


e Interviewer can prescreen 
participant to ensure he 
or she fits the population 
profile. 


e CAPI—computer-assisted 
personal interviewing: Re- 
sponses can be entered into 
a portable microcomputer to 
reduce error and cost. 


e High costs. 


e Need for highly trained 
interviewers. 


e Longer period needed in the 
field collecting data. 

e May be wide geographic 
dispersion. 


e Follow-up is labor-intensive. 


e Not all participants are avail- 
able or accessible. 


e Some participants are unwill- 
ing to talk to strangers in 
their homes. 


e Some neighborhoods are 
difficult to visit. 


e Questions may be altered 
or participant coached by 
interviewers. 


226 


>picorofile 


>part Il The Design of Business Research 


Quantitative Interviews by ADM Member Agencies 
by Method of Interview 

70% - 

60% |- 

50% - 

40% - 

30% - 

20% - 

10% 

a 1990 1995 2000 2005 2010 

=== Face-to-face interviews = = Telephone interviews 
=== Mailed questionnaires === Online interviews 


The Internet has had a major influence on method of data collection over the last decade. ADM Arbeitskreis Deutscher Markt- 
und Sozialforschungsinstitute e.V. is a business association that represents the interests of private-sector market and social 
research agencies in Germany. Its members’ methodology shift is mirrored by researchers in other countries around the world. 
As online interviews and surveys continue to mirror the population contacted via other means, and given its strengths of speed, 
cost, interactivity, and visual and audio stimulation, this methodology is expected to continue to grow rapidly. www.adm-ev.de 


> Self-Administered Surveys 


The self-administered questionnaire is ubiquitous in modern living. You have experienced service 
evaluations of hotels, restaurants, car dealerships, and transportation providers. Often, a short question- 
naire is left to be completed by the participant in a convenient location or is packaged with a prod- 
uct. User registrations, product information requests in magazines, warranty cards, online surveys of 
employee motivation, the MindWriter CompleteCare study, and the Albany Clinic study are examples 
of self-administered surveys. Self-administered mail surveys are delivered not only by the U.S. Postal 
Service but also via fax and courier service. Other delivery modalities include computer-delivered and 
intercept studies. 


Evaluation of the Self-Administered Survey 


Nowhere has the computer revolution been felt more strongly than in the area of the self-administered 
survey. Computer-delivered self-administered questionnaires (also labeled computer-assisted self- 
interviews, or CASIs) use organizational intranets, the Internet, or online services via tablet and mobile 
devices to reach their participants. Participants may be targeted (as when BizRate, an online e-business 
rating service, sends an e-mail to a registered e-purchaser to participate in a survey following the comple- 
tion of their order) or self-selecting (as when a computer screen pop-up window offers a survey to an indi- 
vidual who clicks on a particular website or when a potential participant responds to a postcard or e-mail 
inquiry looking for participants). The questionnaire and its managing software may reside on the com- 
puter or its network, on the cloud, or both may be sent to the participant by mail—disk-by-mail (DBM) 
survey. Data from Department of Commerce’s Economics and Statistics Administration and National 
Telecommunications and Information Administration (NTIA) show 78.38 percent’ of U.S. households 
are actively online, while that number drops below 50 percent’? for poor households. Almost one-third of 
Americans are not accessing broadband service at home.'! Is it any wonder, then, that researchers while 
embracing online surveys recognize that while such surveys have numerous advantages that the sample 
drawn for such a survey may not be representative of the U.S. population. See Exhibit 10-6. 
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Exhibit 10-6 The Web as a Survey Research Venue 


Web Advantages Example 


Short turnaround of results; results are 
tallied as participants complete surveys. 


Ability to use visual stimuli. 


Ability to do numerous surveys over time. 


Ability to attract participants who wouldn’t 
participate in another research project, 
including international participants. 


Participants feel anonymous. 


Shortened turnaround from questionnaire 
draft to execution of survey. 


Web Disadvantages 
(and emerging solutions) 


Recruiting the right sample is costly and 
time-consuming; unlike phone and mail 
sample frames, no lists exist and must 
be built. 

(Firms like Toluna and Survey Samples Inc. 
now provide samples built from panels 
of Internet users who have indicated an 
interest in participating in online surveys.) 


Converting surveys to the Web can be 
expensive. 

(Firms like Qualtric Labs with its SurveyPro 
software and Apian with its Perseus 
software for wireless surveys and in- 
tranet surveys have made the process 
of going from paper to Internet much 
easier.) 


It takes technical as well as research skill to 
field a Web survey. 

(Numerous firms now offer survey hosting 
services, e.g., SurveyMonkey.com.) 


While research is more compatible with 
numerous browsers, the technology 
isn’t perfect. 

(Some survey hosting services use initial 
survey screen questions that identify 
the browser and system specifications 
and deliver the survey in the format 
most compatible with the participant’s 
system.) 


A soft-drink manufacturer got results from a Web survey in just five days. 


e Florida’s tourism office used eye movement tracking to enhance its website and 
improve its billboard and print ads. 


e One major advertising agency is conducting Web research using virtual supermarket 
aisles that participants wander through, reacting to client products and promotions. 


e LiveWorld has developed a packaging study showing more than 75 images of labels 
and bottle designs. 


A printer manufacturer did seven surveys in six months during the development of one 
of its latest products. 


An agricultural equipment manufacturer did a study using two-way pagers provided 
free to farmers to query users about its equipment—participants usually unavailable 
by phone or PC. 


Anonymity was the necessary ingredient for a study on impotence conducted by a 
drug manufacturer. 


A Hewlett-Packard survey using Greenfield Online’s QuickTake took two weeks to 
write, launch, and field—not the standard three months using non-Web venues. 


Example 


TalkCity, working for Whitton Associates and Fusion5, set up a panel of 3,700 teens 
for a survey to test new packaging for a soft drink using phone calls, referrals, 
e-mail lists, banner ads, and website visits. It drew a sample of 600 for the research. 
It cost more than $50,000 to set up the list. 


LiveWorld’s teen study cost $50,000 to $100,000 to set up, plus additional fees with 
each focus group or survey. The total price tag was several hundred thousand 
dollars. 


A 10- to 15-minute survey can take up to five days of technical expertise to field and 
test. 


A well-known business magazine did a study among a recruited sample only to have 
the survey abort on question 20 of a longer study. 


Source: These examples are drawn from the personal experience of the authors, as well as from Noah Shachtman, “Why the Web Works as 
a Market Research Tool,” AdAge.com, Summer 2001 (http://adage.com/tools2001). 
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Intercept surveys—at malls, conventions, state fairs, vacation destinations, even busy city street 
corners—may use a traditional paper-and-pencil questionnaire or a computer-delivered survey via an 
iPad, or netbook computer, or a kiosk. The respondent participates without interviewer assistance, 
usually in a predetermined environment, such as a room in a shopping mall. All modes have special 
problems and unique advantages (as shown in Exhibit 10-5). 

Because computer-delivered surveys, especially those delivered via the Internet, are in their in- 
fancy, much of what researchers know about self-administered surveys has been learned from ex- 
periments conducted with mail surveys and from personal experience. So as we explore the strengths 
and weaknesses of the various self-administered survey methods, we will start with this body of 
knowledge. 


Costs 


Self-administered surveys of all types typically cost less than surveys via personal interviews. This is 
true of mail surveys, as well as of both computer-delivered and intercept surveys. Telephone and mail 
costs are in the same general range, although in specific cases either may be lower. The more geo- 
graphically dispersed the sample, the more likely it is that self-administered surveys via computer or 
mail will be the low-cost method. A mail or computer-delivered study can cost less because it is often 
a one-person job. And computer-delivered studies (including those that employ interviewer-participant 
interaction) eliminate the cost of printing surveys, a significant cost of both mail studies and personal 
interviewing employing printed surveys. The most significant cost savings with computer-delivered 
surveys involve the much lower cost of pre- and postnotification (often done by mail or phone when 
other self-administered surveys are involved), the lower data entry costs (especially with self-admin- 
istered computer-delivered surveys), as well as the lower per-participant survey delivery cost of very 
large studies.” 


Sample Accessibility 


One asset to using mail self-administered surveys is that researchers can contact participants who 
might otherwise be inaccessible. Some groups, such as major corporate executives and physicians, are 
difficult to reach in person or by phone, as gatekeepers (secretaries, office managers, and assistants) 
limit access. But researchers can often access these special participants by mail or computer. When the 
researcher has no specific person to contact—say, in a study of corporations—the mail or computer- 
delivered survey may be routed to the appropriate participant. Additionally, the computer-delivered 
survey can often reach samples that are identified in no way other than their computer and Internet 
use, such as the users of a particular online game or those who have shopped with a particular online 
retailer. 


Time Constraints 


Although intercept studies still pressure participants for a relatively quick response, in a mail survey 
the participant can take more time to collect facts, talk with others, or consider replies at length than 
is possible in a survey employing the telephone or in a personal interview. Computer-delivered stud- 
ies, especially those accessed via e-mail links to the Internet, often have time limitations on both 
access and completion once started. And once started, computer-delivered studies usually cannot 
be interrupted by the participant to seek information not immediately known. One recent computer- 
delivered study sponsored by Procter & Gamble, however, asked of participants (who used skin 
moisturizers) the actual duration of time that the participant spent applying the product to various 
skin areas following a bath or shower. These questions came in the middle of a fairly lengthy survey. 
The participant was encouraged to discontinue the survey, time his or her moisturizer application 
following the next bath or shower, and return to the survey via a link and personal code with detailed 
responses.'3 
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Mixed-mode research describes research where different participants in the same study will access a questionnaire by differ- 
ent methods (e.g., phone interview, online via computer, online via mobile device, personal interview, etc.). In a recent study 
by Survey Sampling Inc., the long-time leader in sampling set out to determine whether the mode created relevant differences. 
When asking neutral behavioral questions (like whether they shopped for groceries) there was little difference created by the 
different modes. When assessing behaviors where social desirability error was likely to be present, like whether they drank 
alcohol, those who participated by self-completion mode (e.g., online) showed less social desirability error, than those where 
an interviewer was involved. Reduction of social-desirability error and noncoverage error are seen as the strengths of self- 
completion modes like the online survey. www.surveysampling.com 


Anonymity 


Mail surveys are typically perceived as more impersonal, providing more anonymity than the other 
communication modes, including other methods for distributing self-administered questionnaires. 
Computer-delivered surveys still enjoy that same perceived anonymity, although increased concerns 
about privacy may erode this perception in the future.'* 


Topic Coverage 


A major limitation of self-administered surveys concerns the type and amount of information that can 
be secured. Researchers normally do not expect to obtain large amounts of information and cannot 
probe deeply into topics. Participants will generally refuse to cooperate with a long and/or complex 
mail, computer-delivered, or intercept questionnaire unless they perceive a personal benefit. Returned 
mail questionnaires with many questions left unanswered testify to this problem, but there are also 
many exceptions. One general rule of thumb is that the participant should be able to answer the ques- 
tionnaire in no more than 10 minutes—similar to the guidelines proposed for telephone studies. On 
the other hand, one study of the general population delivered more than a 70 percent response to a 
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questionnaire calling for 158 answers.'* Several early studies of computer-delivered surveys show that 
participants indicate some level of enjoyment with the process, describing the surveys as interesting 
and amusing.’® The novelty of the process, however, is expected to decline with experience, and recent 
declines in Web and e-mail survey response rates seem to be supporting this expectation. 


Maximizing Participation in the 
Self-Administered Survey 


To maximize the overall probability of response, attention must be given to each point of the survey 
process where the response may break down." For example: 


e The wrong address, e-mail or postal, can result in nondelivery or nonreturn. 


e The envelope or fax cover sheet may look like junk mail and be discarded without being opened, 
or the subject line on e-mail may give the impression of spam and not encourage that the e-mail 
be opened. 


¢ Lack of proper instructions for completion may lead to nonresponse. 


e The wrong person may open the envelope or receive the fax or e-mail and fail to call it to the 
attention of the right person. 


e A participant may find no convincing explanation or inducement for completing the survey and 
thus discard it. 


e A participant may temporarily set the questionnaire aside or park it in his or her e-mail in-box 
and fail to complete it. 


e The return address may be lost, so the questionnaire cannot be returned. 


Thus, efforts should be directed toward maximizing the overall probability of response. One approach, 
the Total Design Method (TDM), suggests minimizing the burden on participants by designing ques- 
tionnaires that:'* 


e Are easy to read. 

° Offer clear response directions. 

e Include personalized communication. 

e Provide information about the survey via advance notification. 
e Encourage participants to respond.'? 


More than 200 methodological articles have been published on efforts to improve response rates. Few 
approaches consistently showed positive response rates.”” However, several practical suggestions 
emerge from the conclusions:”! 


e Preliminary or advance notification of the delivery of a self-administered questionnaire increases 
response rates. 


e Follow-ups or reminders after the delivery of a self-administered questionnaire increase 
response rates. 


° Clearly specified return directions and devices (e.g., response envelopes, especially postage- 
stamped) improve response rates. 


¢ Monetary incentives for participation increase response rates. 

e Deadline dates do not increase response rates but do encourage participants to respond sooner. 

e A promise of anonymity, although important to those who do respond, does not increase response rates. 
e An appeal for participation is essential. 


Self-Administered Survey Trends 


Computer surveying is surfacing at trade shows, where participants complete questionnaires while 
making a visit to a company’s booth. Continuous tabulation of results provides a stimulus for attend- 
ees to visit a particular exhibit as well as gives the exhibitor detailed information for evaluating the 
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productivity of the show. This same technology easily transfers to other situations where large groups 
of people congregate. 

Companies are now using intranet capabilities to evaluate employee policies and behavior. Ease of 
access to electronic mail systems makes it possible for both large and small organizations to use com- 
puter surveys with both internal and external participant groups. Many techniques of traditional mail 
surveys can be easily adapted to computer-delivered questionnaires (e.g., follow-ups to nonparticipants 
are more easily executed and are less expensive). 

It is not unusual to find registration procedures and full-scale surveying being done on World Wide 
Web sites. University sites are asking prospective students about their interests, and university depart- 
ments are evaluating current students’ use of online materials. A short voyage on the Internet reveals 
organizations using their sites to evaluate customer service processes, build sales-lead lists, evaluate 
planned promotions and product changes, determine supplier and customer needs, discover interest in 
job openings, evaluate employee attitudes, and more. Advanced and easier-to-use software for design- 
ing Web questionnaires is no longer a promise of the future—it’s here. 

The Web-based questionnaire, a measurement instrument both delivered and collected via the 
Internet, has the power of computer-assisted telephone interview systems, but without the expense of 
network administrators, specialized software, or additional hardware. As a solution for Internet or in- 
tranet websites, you need only a personal computer and Web access. Most software products are wizard 
driven with design features that allow custom survey creation and modification. 

Two primary options are proprietary solutions offered through research firms and off-the-shelf soft- 
ware designed for researchers who possess the knowledge and skills we describe here and in Chap- 
ters 11, 12, and 13. With fee-based services, you are guided (often online) through problem formulation, 
questionnaire design, question content, response strategy, and wording and sequence of questions. The 
supplier’s staff generates the questionnaire HTML code, hosts the survey at their server, and provides 
data consolidation and reports. Off-the-shelf software is a strong alternative.” The advantages of these 
software programs are: 


* Questionnaire design in a word processing environment. 

e Ability to import questionnaire forms from text files. 

e A coaching device to guide you through question and response formatting. 
¢ Question and scale libraries. 

e Automated publishing to a Web server. 

¢ Real-time viewing of incoming data. 

e Ability to edit data in a spreadsheet-type environment. 

¢ Rapid transmission of results. 


e Flexible analysis and reporting mechanisms. 


Ease of use is not the only influence pushing the popularity of web-based instruments. Cost is a 
major factor. A Web survey is much less expensive than conventional survey research. Although fees 
are based on the number of completions, the cost of a sample of 100 might be one-sixth that of a con- 
ventional telephone interview. Bulk mailing and e-mail data collection have also become more cost- 
effective because any instrument may be configured as an e-mail questionnaire. 

The computer-delivered survey has made it possible to use many of the suggestions for increasing 
participation. Once the computer-delivered survey is crafted, the cost of redelivery via computer is 
very low. Preliminary notification via e-mail is both more timely and less costly than notification for 
surveys done by phone or mail. The click of a mouse or a single keystroke returns a computer-delivered 
study. Many computer-delivered surveys use color, even color photographs, within the survey struc- 
ture. This is not a cost-effective option with paper surveys. And video clips—never an option with a 
mail survey—are possible with a computer-delivered survey. In addition, e-currencies have simplified 
the delivery of monetary and other incentives. However, employing all the stimulants for participation 
cannot overcome technology snafus. These glitches are likely to continue to plague participation as 
long as researchers and participants use different computer platforms, operating systems, and software. 

While web- and e-mail-based self-administered surveys have certainly caught the lion’s share of 
business attention in the last few years, the tried-and-true methods of telephone and personal interviews 
still have their strengths—and their advocates in the research community. 
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Are Cell Phones and Smartphones Ready for Research? 


According to a National Health Interview Survey, 31.6 percent 
of U.S. adults live in wireless-phone-only households. This 
number is three times the number of such households in 2006. 
With landline phone coverage on the decline in many developed 
countries, researchers are using sampling techniques and sur- 
vey design to address issues and take advantage of the oppor- 
tunities that wireless phones offer. 

According to Andy Peytchev, PhD, survey methodolo- 
gist with Research Triangle Institute (RTI International) in North 
Carolina, multimethod phone studies, where some participants 
are reached via landlines and others via cell phones, are be- 
coming the norm rather than the exception to achieve a national 
probability sample. Landline data can be weighted according to 
census population parameters. But not all problems can be fixed 
by weighting. “Youth landline responses versus youth cell phone 
responses, for example, are different. We don’t know exactly 
why, just that they are different. You would have potential for 
undercoverage errors if you just used cell phones or landlines.” 

Cell phone surveys offer unique challenges. Many partici- 
pants want to know why they are being contacted on their cell 
phone. So RTI International has changed its standard phone in- 
troduction. Interviewers inform cell phone participants that they 
know they are being contacted via their cell phone and why. RT| 


International also offers cell phone respondents an incentive to 
continue the survey that is sufficient to show its appreciation and 
recognize the cost of cell phone minute charges. Cell phone sur- 
veys also require additional questions. RTI International trains its 
interviewers to inquire, “Are you driving right now?” If participants 
are driving, surveyors ask for an alternative time to contact the 
participant, and then disconnect the call. Interviewers may also 
ask cell phone participants whether they are in a safe place. 

Little research has been done on self-administered smart- 
phone surveys, but Peytchev is interested in the opportunities. 
Smartphone surveys can include images—both those the par- 
ticipants take and share and those shared by the researcher. 
But Peytchev cautions, “Images can distort the meaning of the 
question. Everything you present to the participant is seen as 
information.” And images collected from participants have to be 
analyzed and interpreted. Some RT! experiments have revealed 
that participants are reluctant to complete text-box responses 
and that if such a response device is included, cell phone par- 
ticipants are more likely to choose nonsense answers to avoid 
texting. This adds a new twist to survey research on the age-old 
issue of just what questions to ask. 


www.rti.org; www.cdc.gov/nchs 


> Survey via Telephone Interview 


The telephone survey is still the workhorse of survey research. With the high level of telephone ser- 
vice penetration in the United States and the European Union, access to participants through low-cost, 
efficient means has made telephone interviewing a very attractive alternative for researchers. Nielsen 
Media Research uses thousands of calls each week to determine television viewing habits, and Arbitron 
does the same for radio listening habits. Pollsters working with political candidates use telephone sur- 
veys to assess the power of a speech or a debate during a hotly contested campaign. Numerous firms 
field phone omnibus studies each week. Individual questions in these studies are used to capture every- 
thing from people’s feeling about the rise in gasoline prices to the power of a celebrity spokesperson in 
an advertising campaign or the latest teenage fashion trend. 


Evaluation of the Telephone Interview 


Of the advantages that telephone interviewing offers, probably none ranks higher than its moderate 
cost. One study reports that sampling and data collection costs for telephone surveys can run from 45 
to 64 percent lower than costs for comparable personal interviews.”? Much of the savings comes from 
cuts in travel costs and administrative savings from training and supervision. When calls are made 
from a single location, the researcher may use fewer, yet more skilled, interviewers. Telephones are 
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PEW Reseach Center Reveals Declining Telephone Response Rates 


Percent Response 
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Data are provided by the PEW Research Center. Rates computed according to the American Association for Public Opinion 
Research (AAPOR) standard definition for a completed telephone survey where an adult was reached and agreed to participate; 


rates are typical for surveys conducted each year. 


especially economical when callbacks to maintain precise sampling requirements are necessary and 
participants are widely scattered. Long-distance service options make it possible to interview nation- 
ally at a reasonable cost. 

Telephone interviewing can be combined with immediate entry of the responses into a data file 
by means of terminals, personal computers, or voice data entry. This brings added savings in time 
and money. The computer-assisted telephone interview (CATID) is used in research organiza- 
tions throughout the world. A CATI facility consists of acoustically isolated interviewing carrels 
organized around supervisory stations. The telephone interviewer in each carrel has a personal 
computer or terminal that is networked to the phone system and to the central data processing 
unit. A software program that prompts the interviewer with introductory statements, qualifying 
questions, and precoded questionnaire items drives the survey. These materials appear on the in- 
terviewer’s monitor. CATI works with a telephone number management system to select numbers, 
dial the sample, and enter responses. One facility, the Survey Research Center at the University of 
Michigan, consists of 55 interviewer carrels with 100 interviewers working in shifts from 8 a.m. 
to midnight (EST) to call nationwide. When fully staffed, it produces more than 10,000 interview 
hours per month.”4 

Another means of securing immediate response data is the computer-administered telephone 
survey. Unlike CATI, there is no human interviewer. A computer calls the phone number, conducts 
the interview, places data into a file for later tabulation, and terminates the contact. The questions are 
voice-synthesized, and the participant’s answers and computer timing trigger continuation or discon- 
nect. Several modes of computer-administered surveys exist, including touch-tone data entry (TDE); 
voice recognition (VR), which recognizes a limited vocabulary—usually yes/no responses; and auto- 
matic speech recognition (ASR) for recognizing and recording a wide range of verbal responses. CATI 
is often compared to the self-administered questionnaire and offers the advantage of enhanced partici- 
pant privacy. One study showed that the noncontact rate for this electronic survey mode is similar to 
that for other telephone interviews when a random phone list is used. It also found that rejection of this 
mode of data collection affects the refusal rate (and thus nonresponse error) because people hang up 
more easily on a computer than on a human.” The noncontact rate is a ratio of potential but unreached 
contacts (no answer, busy, answering machine or voice mail, and disconnects but not refusals) to all 
potential contacts. 

The refusal rate refers to the ratio of contacted participants who decline the interview to all poten- 
tial contacts. New technology, notably call-filtering systems in which the receiver can decide whether 
a call is answered based on caller identity, is expected to increase the noncontact rate associated with 


234 


>part Il The Design of Business Research 


telephone surveys. According to a 2005 study, survey refusal rates have been steadily increasing, by 
approximately 1.5 percent points each year and that trend is expected to continue.”° Another study 
noted that “positive attitudes [about participating in surveys] are declining, while negative perceptions 
are increasing.” 

When compared to either personal interviews or mail self-administered surveys, the use of tele- 
phones brings a faster completion of a study, sometimes taking only a day or so for the fieldwork. When 
compared to personal interviewing, it is also likely that interviewer bias, especially bias caused by the 
physical appearance, body language, and actions of the interviewer, is reduced by using telephones. 

Finally, behavioral norms work to the advantage of telephone interviewing. If someone is present, a 
ringing phone is usually answered, and it is the caller who decides the purpose, length, and termination 
of the call.”8 

There are also disadvantages to using the telephone for research. A skilled researcher will evaluate 
the use of a telephone survey to minimize the effect of these disadvantages: 


e Inaccessible households (no telephone service or no/low contact rate). 
e Inaccurate or nonfunctioning numbers. 

e Limitation on interview length (fewer measurement questions). 

e Limitations on use of visual or complex questions. 

e Ease of interview termination. 

° Less participant involvement. 


e Distracting physical environment. 


Inaccessible Households 


Approximately 94 percent of all U.S. households have access to telephone service.*? On the sur- 
face, this should make telephone surveys a prime methodology for communication studies. How- 
ever, several factors reduce such an enthusiastic embrace of the methodology. Rural households 
and households with incomes below the poverty line remain underrepresented in telephone stud- 
ies, with phone access below 75 percent.*° More households are using filtering devices and ser- 
vices to restrict access, including caller ID, privacy manager, Tele-Zapper, and unlisted numbers 
(estimated between 22 and 30 percent of all household phone numbers).*! Meanwhile, the number 
of inaccessible individuals continues to increase as cellular/wireless phone use increases. From 
1985 to 2011, the number of U.S. wireless telecommunication subscribers grew from 203.6 thou- 
sand* to 327.6 million.** Many of these numbers are unlisted or possess screening or filtering 
services. Additionally, people’s use of phone modems to access the Internet makes household 
lines ring busy for long periods of time. Recent FCC filings indicate that fewer than 15 percent of 
U.S. households have second telephone lines, required for simultaneous Internet access.** Effec- 
tive May 2004 federal wireless local-number portability legislation made it possible for subscrib- 
ers to take their wired phone number to their wireless phone service (or the reverse) or to shift 
their wireless service between carriers without losing their wireless number. Thus, the guidelines 
for identifying the physical location of a phone by its number—and, in turn, the location of its 
owner—no longer apply.* 

These causes of variations in participant availability by phone can be a source of bias. A random 
dialing procedure is designed to reduce some of this bias. Random dialing normally requires choos- 
ing phone exchanges or exchange blocks and then generating random numbers within these blocks for 
calling.*° Of course, just reaching a household doesn’t guarantee its participation. 


Inaccurate or Nonfunctioning Numbers 


One source says the highest incidence of unlisted numbers is in the West, in large metropolitan areas, 
among nonwhites, and for persons between 18 and 34 years of age.*’ Several methods have been devel- 
oped to overcome the deficiencies of directories; among them are techniques for choosing phone num- 
bers by using random dialing or combinations of directories and random dialing.** However, increasing 
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In several polls related to the 
public’s understanding of the 
Do Not Call registry, Harris 
Interactive found that although 
the registry was working to 
reduce undesired telemarketing 
calls (91 percent reported 
fewer or no calls), there was 
still confusion about whether 
survey calls were restricted. 

In one study, Harris found 
that 42 percent of U.S. adults 
erroneously thought registering 
for the national Do Not Call 
registry would ban telephone 
survey calls as well. Five years 
later, Harris found that 

63 percent of those who 
actually registered did not 
know whether survey 
researchers were allowed to 
call, while only 24 percent 
knew that registration would 
not block survey calls. Since 
the Do Not Call registry was 
established, 70 percent of 
those who are registered have 
been surveyed by phone. But 
many (29 percent) still do not 
know that registration expires 
and must be renewed. What's 
the true measure of success 
a of this program? That fully 


Farewe ll Old Friend 96 percent have renewed or 


will renew their registration. 


www.harrisinteractive.com 


There are now more than 50 million consumer phone numbers in the 
National Do Not Call Registry — and it’s growing every day. To complicate 
matters, almost half* of all Americans incorrectly believe that telephone 
survey calls are now banned by the Do Not Call rules. 


The future of telephone survey work is in doubt. But there is no doubt 
that viable, accurate data collection alternatives exist. 


Call Harris Interactive today to learn more — we'll be happy to = 
answer thelphone; @ Interactive 


MARKET RESEARCH 
“42% of all U.S, adults think that the registry applies to calls to conduct surveys about products 
and services — The Harris Poll® - September 4, 2003 


www.harrisinteractive.com/DNC Tel 877.919.4765 


©2005, Harris Interactive Inc. All rights reserved. EOE M/F/D/V 04.05 


demand for multiple phone lines by both households and individuals has generated new phone area 
codes and local exchanges. This too increases the inaccuracy rate. 


Limitation on Interview Length 


A limit on interview length is another disadvantage of the telephone survey, but the degree of this 
limitation depends on the participant’s interest in the topic. Ten minutes has generally been thought 
of as ideal, but interviews of 20 minutes or more are not uncommon. One telephone survey sponsored 
by Kraft lasted approximately 30 minutes. It was designed to judge the willingness of sample issue 
recipients to subscribe to a prototype magazine, food&family. The survey also measured the effective- 
ness of the sample issue of the magazine to deliver purchase intent for Kraft products featured in the 
recipes contained therein.*? In another study, interviews ran for one and a half hours in a survey of 
long-distance services.” 
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Limitations on Use of Visual or Complex Questions 


The telephone survey limits the complexity of the survey and the use of complex scales or measure- 
ment techniques that is possible with personal interviewing, CASI, or WWW surveys. For example, 
in personal interviews, participants are sometimes asked to sort or rank an array of cards containing 
different responses to a question. For participants who cannot visualize a scale or other measurement 
device that the interview is attempting to describe, one solution has been to employ a nine-point 
scaling approach and to ask the participant to visualize it by using the telephone dial or keypad.*! 
In telephone interviewing it is difficult to use maps, illustrations, and other visual aids. In some in- 
stances, however, interviewers have supplied these prior to a prescheduled interview via fax, e-mail, 
or the Internet. 


Ease of Interview Termination 


Some studies suggest that the response rate in telephone studies is lower than that for comparable 
face-to-face interviews. One reason is that participants find it easier to terminate a phone interview. 
Telemarketing practices may also contribute. Public reaction to investigative reports of wrongdoing 
and unethical behavior within telemarketing activities places an added burden on the researcher, who 
must try to convince a participant that the phone interview is not a pretext for soliciting contributions 
(labeled frugging—fund-raising under the guise of research) or selling products (labeled sugging— 
sales under the guise of research). 


Less Participant Involvement 


Telephone surveys can result in less thorough responses, and persons interviewed by phone find the ex- 
perience to be less rewarding than a personal interview. Participants report less rapport with telephone 
interviewers than with personal interviewers. Given the growing costs and difficulties of personal in- 
terviews, it is likely that an even higher share of surveys will be by telephone in the future. Thus, it 
behooves researchers using telephone surveys to attempt to improve the enjoyment of the interview. 
One authority suggests: 


We need to experiment with techniques to improve the enjoyment of the interview by the participant, maximize the 
overall completion rate, and minimize response error on specific measures. This work might fruitfully begin with efforts 
at translating into verbal messages the visual cues that fill the interaction in a face-to-face interview: the smiles, frowns, 
raising of eyebrows, eye contact, etc. All of these cues have informational content and are important parts of the per- 
sonal interview setting. We can perhaps purposefully choose those cues that are most important to data quality and 
participant trust and discard the many that are extraneous to the survey interaction.” 


Changes in the Physical Environment 


Replacement of home or office phones with cellular and wireless phones also raises concerns. In regard 
to telephone surveys, researchers are concerned about the changing environment in which such surveys 
might be conducted, the resulting quality of data collected under possibly distracting circumstances— 
at a busy intersection, in the midst of weekly shopping in a congested grocery aisle, at the local high 
school basketball tournament—and the possible increase in refusal rates. 


Telephone Survey Trends 


Future trends in telephone surveying bear watching. Answering machines or voice-mail services pose 
potentially complex response rate problems since they are estimated to have substantial penetration in 
American households. Previous research discovered that most such households are accessible; the sub- 
sequent contact rate was greater in answering-machine households than in no-machine households and 
about equal with busy-signal households. Other findings suggested that (1) individuals with answering 
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Do telephone surveys and interviews fit in today’s Web-survey 
dominant research landscape? Anderson Analytics, a Stamford 
(CT) business intelligence and marketing research company, re- 
cently teamed up with BigEars, a New Zealand-based company 
specializing in fully automated telephone surveys and interviews, 
to conduct a survey among college students about their cell 
phones. The hybrid study employed both an online survey and 
the automated telephone survey method. 

The BigEars operates much like a Web-based survey tool. 
The difference is that the respondent answers over the phone, 
typically via an 800 number. “By eliminating the human inter- 
viewer from the call, we allow the caller to participate whenever it 
suits them, rather than when it suits us,” shared Tom Anderson, 
managing partner of Anderson Analytics. 

The survey results indicated that the main advantage of 
using telephone surveys is its ability to encourage longer and 
more robust responses to open-ended questions. According 


survey. In addition, the voice recording offered opportunities for 
in-depth qualitative analysis; emotion and inflection in individual 
voice clips were used to examine the outliers in the study. “Being 
able to listen to the actual voices of the students, rather than 
simply coding or reading their responses, allows for a whole new 
dimension of analysis and confidence in the findings” said Tom 
Anderson. 

“With this new hybrid methodology, if you want reaction to 
an event or transaction, such as a visit to a store, you can cap- 
ture your data immediately, while it’s fresh in the person’s mind,” 
said Jesse Chen, senior consultant and developer at Anderson 
Analytics. “Some people are more comfortable on the Web, and 
some are more comfortable on the phone—by catering to these 
differences you can broaden participation.” 

So how do college students feel about their cell phones? 
You can see the top-line report at http://www.andersonanalytics 
.com/reports/AndersonAnalyticsBigEars. ppt. 


to Mark Forsyth, managing director of BigEars, “Talking isn’t 
work.” Answers given to open-ended questions over the phone 
were 15 percent longer than answers typed in the parallel online 


www.andersonanalytics.com; www.yourbigears.com 


machines were more likely to participate, (2) machine use was more prevalent on weekends than on 
weekday evenings, and (3) machines were more commonplace in urban than in rural areas. 

Voice-mail options offered by local phone service providers have less market penetration but are 
gaining increasing acceptance. Questions about the sociodemographics of users and nonusers and the 
relationship of answering-machine/voice-mail technology to the rapid changes in the wireless market 
remain to be answered.*? Caller identification technology, the assignment of facsimile machines or 
computer modems to dedicated phone lines, and technology that identifies computer-automated dialers 
and sends a disconnect signal in response are all expected to have an impact on the noncontact rate of 
phone interviews. 

The variations among the 60 telephone companies’ services and the degree of cooperation that will 
be extended to researchers are also likely to affect noncontact rates. There is also concern about the ways 
in which random dialing can be made to deal with nonworking and ineligible numbers.“ But arguably 
no single threat poses greater danger than the government-facilitated Do Not Call registry initiated in 
2003 by the Federal Trade Commission* and amended by the Do-Not-Call Improvement Act of 2007.*° 
More than 209 million U.S. household and cell numbers are now registered.*° Although currently survey 
researchers are exempt from its restrictions, customer confusion about the distinction between research 
and telemarketing is likely to cause an increase in the nonresponse rate. Telemarketers might be the 
catalyst, but legitimate research will suffer. 


> Survey via Personal Interview 


A survey via personal interview is a two-way conversation between a trained interviewer and a par- 
ticipant. With her poor eyesight and the problems of question clarity, a personal interview, rather than 
the intercept/self-administered questionnaire, might have been a preferable communication method for 
Edna at the Albany Outpatient Laser Clinic. 
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Evaluation of the Personal Interview Survey 


There are real advantages as well as clear limitations to surveys via personal interview. The greatest 
value lies in the depth of information and detail that can be secured. It far exceeds the information se- 
cured from telephone and self-administered studies via mail or computer (both intranet and Internet). 
The interviewer can also do more things to improve the quality of the information received than is 
possible with another method. 

The absence of assistance in interpreting questions in the Albany Clinic study was a clear weak- 
ness that would have been improved by the presence of an interviewer. Interviewers can note condi- 
tions of the interview, probe with additional questions, and gather supplemental information through 
observation. Edna was obviously in good spirits and very relaxed after she and her fellow patients had 
critiqued the questionnaire. This attitude would have been observed and noted by an interviewer. Of 
course, we’re hopeful that the interviewer would correctly interpret laughter as a sign of humor and not 
as a negative attitude, as did the admissions clerk. 

Human interviewers also have more control than other kinds of communication studies. They can 
prescreen to ensure the correct participant is replying, and they can set up and control interviewing con- 
ditions. They can use special scoring devices and visual materials, as is done with a computer-assisted 
personal interview (CAPI). Interviewers also can adjust the language of the interview as they observe 
the problems and effects the interview is having on the participant. 

With such advantages, why would anyone want to use any other survey method? Probably the 
greatest reason is that personal interviewing is costly, in terms of both money and time. A survey via 
personal interview may cost anywhere from a few dollars to several hundred dollars for an interview 
with a hard-to-reach person. Costs are particularly high if the study covers a wide geographic area 
or has stringent sampling requirements. An exception to this is the survey via intercept interview 
that targets participants in centralized locations such as retail malls or, as with Edna, in a doctor’s 
office. Intercept interviews reduce costs associated with the need for several interviewers, training, and 
travel. Product and service demonstrations also can be coordinated, further reducing costs. Their cost- 
effectiveness, however, is offset when representative sampling is crucial to the study’s outcome. The 
intercept survey would have been a possibility in the Albany Clinic study, although more admissions 
clerks would likely have been needed if volunteers were not available to perform this task. You will 
find tips on intercept surveys on the text website. 

Costs have risen rapidly in recent years for most communication methods because changes in the 
social climate have made personal interviewing more difficult. Many people today are reluctant to talk 
with strangers or to permit strangers to visit in their homes. Interviewers are reluctant to visit unfamil- 
iar neighborhoods alone, especially for evening interviewing. Finally, results of surveys via personal 
interviews can be affected adversely by interviewers who alter the questions asked or in other ways 
bias the results. As Edna and her friends discussed the Albany Clinic survey, they each applied their 
own operational definitions to the concepts and constructs being asked. This confusion created a bias 
that might have been eliminated by a well-trained interviewer. Interviewer bias, identified as one of the 
three major sources of error in Exhibit 10-3, was discussed earlier in this chapter. If we are to overcome 
these deficiencies, we must appreciate the conditions necessary for interview success. 


> Selecting an Optimal Survey Method 


The choice of a communication method is not as complicated as it might first appear. By comparing 
your research objectives with the strengths and weaknesses of each method, you will be able to choose 
one that is suited to your needs. The summary of advantages and disadvantages of personal interviews, 
telephone interviews, and self-administered questionnaires presented in Exhibit 10-5 should be useful 
in making such a comparison. 

When your investigative questions call for information from hard-to-reach or inaccessible partici- 
pants, the telephone interview, mail survey, or computer-delivered survey should be considered. How- 
ever, if data must be collected very quickly, the mail survey would likely be ruled out because of lack of 
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control over the returns. Alternatively, you may decide your objective requires extensive questioning 
and probing; then the survey via personal interview should be considered. 

If none of the choices turns out to be a particularly good fit, it is possible to combine the best charac- 
teristics of two or more alternatives into a hybrid survey. Although this decision will incur the costs of the 
combined modes, the flexibility of tailoring a method to your unique needs is often an acceptable trade-off. 

In the MindWriter study, Jason Henry proposed a post-service email-recruited self-administered 
online survey using a sample of laptop owners who had used the CompleteCare repair service. While 
this research design offers speed in data collection, data entry, and a lower cost than one with signifi- 
cant data entry, it is not without problems. Not all customers will participate when invited, creating 
nonresponse bias. Even though theoretically the online format offers unlimited space for questions, 
only a short survey is likely to encourage participation and full survey completion. Alerting custom- 
ers to the importance of participating in the survey by phone might improve the research design, but 
it would add significantly to the cost of each monthly study. Even including a printed card in each 
returned laptop requesting participation increases the cost significantly. If CompleteCare service users 
receive a damaged laptop, they will not be in the best frame of mind when invited to take the survey, 
and dissatisfaction could lead to a decreased response rate at a time when MindWriter needs to hear 
from these dissatisfied users as much as from the satisfied ones. Jason’s proposal contains a follow-up 
procedure—a telephone survey with invited participants who do not complete the online survey. This 
process will likely decrease nonresponse error while it increases the cost of the design. If most study 
participants are answering measurement questions without assistance, telephone interviewing creates 
the possibility of injecting interviewer bias at an unknown level for at least part of the data. 

In the Albany Clinic study, the researcher could have taken several actions to improve the quality 
of the data. Distributing the questionnaire to the patient’s eye doctor or to the patient (by mail) prior 
to arrival would have increased the accuracy of identifying medications, diagnoses, hospitalizations, 
and so forth. The patient’s eye doctor was in the best position to encourage compliance with the col- 
lection process but was not consulted. Having the patient bring the completed questionnaire to the 
admissions procedure, where the admissions clerk could review the completed instrument for accuracy 
and completeness, would have given the researcher the opportunity to clarify any confusion with the 
questions, concepts, and constructs. Finally, pretesting the instrument with a sample of patients would 
have revealed difficulties with the process and operational definitions. Edna’s concerns could have 
been eliminated before they surfaced. 

Ultimately, all researchers are confronted by the practical realities of cost and deadlines. As 
Exhibit 10-5 suggests, on the average, surveys via personal interview are the most expensive com- 
munication method and take the most field time unless a large field team is used. Telephone surveys 
are moderate in cost and offer the quickest option, especially when CATI is used. Questionnaires ad- 
ministered by e-mail or the Internet are the least expensive. When your desired sample is available via 
the Internet, the Internet survey may prove to be the least expensive communication method with the 
most rapid (simultaneous) data availability. The use of the computer to select participants and reduce 
coding and processing time will continue to improve the cost-to-performance profiles of this method 
in the future. 

Most of the time, an optimal method will be apparent. However, managers’ needs for information 
often exceed their internal resources. Such factors as specialized expertise, a large field team, unique 
facilities, or a rapid turnaround prompt managers to seek assistance from research vendors of survey- 
related services. 


Outsourcing Survey Services 


Commercial suppliers of research services vary from full-service operations to specialty consultants. 
When confidentiality is likely to affect competitive advantage, the manager or staff will sometimes 
prefer to bid only a phase of the project. Alternatively, the organization’s staff members may possess 
such unique knowledge of a product or service that they must fulfill a part of the study themselves. 
Regardless, the exploratory work, design, sampling, data collection, or processing and analysis may be 
contracted separately or as a whole. Most organizations use a request for proposal (RFP) to describe 
their requirements and seek competitive bids (see the sample RFP in Appendix A). 
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Research firms also offer special advantages that their clients do not typically maintain in-house. 
Centralized-location interviewing or computer-assisted telephone facilities may be particularly desir- 
able for certain research needs. A professionally trained staff with considerable experience in similar 
management problems is another benefit. Data processing and statistical analysis capabilities are es- 
pecially important for some projects. Other vendors have specially designed software for interviewing 
and data tabulation.*” Panel suppliers provide another type of research service, with emphasis on lon- 
gitudinal survey work.** By using the same participants over time, a panel can track trends in attitudes 
toward issues or products, product adoption or consumption behavior, and a myriad of other research 
interests. Suppliers of panel data can secure information from personal and telephone interviewing 
techniques as well as from the mail, the Web, and mixed-modes surveys. Diaries are a common means 
of chronicling events of research interest by the panel members. These are mailed back to the research 
organization. Point-of-sale terminals and scanners aid electronic data collection for panel-type partici- 
pant groups. And mechanical devices placed in the homes of panel members may be used to evaluate 
media usage. ACNielsen, Yankelovich Partners, The Gallup Organization, and Harris Interactive all 


manage extensive panels. 


mm 


The communication approach involves surveying or inter- 
viewing people and recording their responses for analysis. 
Communication is accomplished via personal interviews, 
telephone interviews, or self-administered surveys, with 
each method having its specific strengths and weaknesses. 
The optimal communication method is the one that is instru- 
mental for answering your research question and dealing 
with the constraints imposed by time, budget, and human 
resources. The opportunity to combine several survey meth- 
odologies makes the use of the mixed mode desirable in 
many projects. 


Successful communication requires that we seek informa- 
tion the participant can provide and that the participant un- 
derstand his or her role and be motivated to play that role. 
Motivation, in particular, is a task for the interviewer. Good 
rapport with the participant should be established quickly, 
and then the technical process of collecting data should 
begin. The latter often calls for skillful probing to supplement 
the answers volunteered by the participant. Simplicity of di- 
rections and instrument appearance are additional factors to 
consider in encouraging response in self-administered com- 
munication studies. 


Two factors can cause bias in interviewing. One is nonre- 
sponse. It is a concern with all surveys. Some studies show 
that the first contact often secures less than 20 percent of 
the designated participants. Various methods are useful for 
increasing this representation, the most effective being mak- 
ing callbacks until an adequate number of completed inter- 
views have been secured. The second factor is response 
error, which occurs when the participant fails to give a cor- 
rect or complete answer. The interviewer also can contrib- 
ute to response error. The interviewer can provide the main 


solution for both of these two types of errors. 


The self-administered questionnaire can be delivered 

by the U.S. Postal Service, facsimile, a courier service, 

a computer, or an intercept. Computer-delivered self- 
administered questionnaires use organizational intranets, 
the Internet, or online services to reach their participants. 
Participants may be targeted or self-selecting. Intercept 
studies may use a traditional questionnaire or a computer- 
ized instrument in environments where interviewer assis- 
tance is minimal. 

Telephone interviewing remains popular because of the 
diffusion of telephone service in households and the low 
cost of this method compared with personal interviewing. 
Long-distance telephone interviewing has grown. There are 
also disadvantages to telephone interviewing. Many phone 
numbers are unlisted, and directory listings become obso- 
lete quickly. There is also a limit on the length and depth of 
interviews conducted using the telephone. 

The major advantages of personal interviewing are 
the ability to explore topics in great depth, achieve a high 
degree of interviewer control, and provide maximum inter- 
viewer flexibility for meeting unique situations. However, this 
method is costly and time-consuming, and its flexibility can 
result in excessive interviewer bias. 


Outsourcing survey services offers special advantages 

to managers. A professionally trained research staff, 
centralized-location interviewing, focus group facilities, and 
computer-assisted facilities are among them. Specialty 
firms offer software and computer-based assistance for 
telephone and personal interviewing as well as for mail and 
mixed modes. Panel suppliers produce data for longitudinal 
studies of all varieties. 
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pt SASS 


communication approach 218 disk-by-mail (DBM) survey 226 refusal rate 233 


computer-administered telephone 
survey 233 


intercept interview 238 response error 222 


interviewer error 220 self-administered questionnaire 226 


computer-assisted personal interview 


(CAPI) 238 mail survey 226 


survey 218 


noncontact rate 233 survey via personal interview 237 


computer-assisted self-interview 


(CASI) 226 nonresponse error 222 


telephone survey 232 


computer-assisted telephone panel 240 Web-based questionnaire 231 


interview (CATI) 233 random dialing 234 
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Terms in Review 
1 Distinguish among response error, interviewer error, and 
nonresponse error. 


e Astudy of applicant requirements, job tasks, and 
performance expectations as part of a job analysis 
of student work-study jobs on a college campus of 
2,000 students, where 1,500 are involved in the work- 
study program. 


2 How do environmental factors affect response rates in 
personal interviews? How can we overcome these environ- 
mental problems? 6 You decide to take a telephone survey of 40 families in the 
721-exchange area. You want an excellent representation 
of all subscribers in the exchange area. Explain how you 


will carry out this study. 


Making Research Decisions 
3 Assume you are planning to interview shoppers in a shop- 
ping mall about their views on increased food prices and 
what the federal government should do about them. In 7 You plan to conduct a mail survey of the traffic managers 


what different ways might you try to motivate shoppers to 
cooperate in your survey? 


In recent years, in-home personal interviews have grown 
more costly and more difficult to complete. Suppose, 
however, you have a project in which you need to talk with 
people in their homes. What might you do to hold down 
costs and increase the response rate? 


In the following situations, decide whether you would use a 


of 1,000 major manufacturing companies across the coun- 
try. The study concerns their company policies regarding 
the payment of moving expenses for employees who are 
transferred. What might you do to improve the response 
rate of such a survey? 


8 A major corporation agrees to sponsor an internal study on 


sexual harassment in the workplace. This is in response to 
concerns expressed by its female employees. How would 


personal interview, telephone survey, or self-administered you handle the following issues: 


questionnaire. Give your reasons. a The communication approach (self-administered, tele- 


a Asurvey of the residents of a new subdivision on why phone, personal interview, and/or mixed). 


they happened to select that area in which to live. You b The purpose: Fact finding, awareness, relationship 
also wish to secure some information about what they building, and/or change. 
like and do not like about life in the subdivision. 


c Participant motivation. 

b A poll of students at Metro University on their prefer- d 
ences among three candidates who are running for 
president of the student government. 


Minimization of response and nonresponse error. 


Bringing Research to Life 
9 Define the appropriate communication study for the Albany 
Outpatient Laser Clinic. 


c Asurvey of 58 wholesale grocery companies, scattered 
over the eastern United States, on their personnel man- 
agement policies for warehouse personnel. 


d Asurvey of financial officers of the Fortune 500 corpora- 
tions to learn their predictions for the economic outlook 
in their industries in the next year. 


From Concept to Practice 
10 Using Exhibit 10-1 as your guide, graph the communica- 
tion study you designed in question 9. 
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From the Headlines some way to improve business performance. Companies 

11. As competition for a scarcer dollar increases, an increas- now expect to achieve substantial improvements in customer 
ing number of companies are taking action by fostering retention and top-line growth from these targeted customer 
customer-centric cultures that enhance satisfaction and drive programs. If you wanted to know what kind of VoC programs 
bottom-line profitability. Research reveals that 96 percent of were being used and which were effective or ineffective, what 
companies are leveraging voice-of-consumer data (VoC) in type of survey would you design and why? 


ee 
ee Akron Children’s Hospital Marcus Thomas LLC Tests Hypothesis for 
Troy-Bilt Creative Development 


Campbell-Ewald Pumps Awareness 


into the American Heart Association Mastering Teacher Leadership 
Can Research Rescue the Red NCRCC: Teeing Up and New Strategic Direction 
Cross? 


& Ohio Lottery: Innovative Research Design Drives 


Covering Kids with Health Care Winning 


of Statistics 


ee Data Development Proofpoint: Capitalizing on a Reporter’s Love 
ee Donatos: Finding the New Pizza 
ee Starbucks, Bank One, and Visa Launch Starbucks 
Duetto Visa 
Inquiring Minds Want to 


Know—NOW! 


ee Lexus SC 430 


*You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 


& USTA: Come Out Swinging 


The Sources and Collection of Data 
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learningobjectives 
After reading this chapter, you should understand . . . 


The distinction between measuring objects, properties, and indicants of properties. 
The similarities and differences between the four scale types used in measurement and when each is used. 
The four major sources of measurement error. 


The criteria for evaluating good measurement. 


6 6 You’re trying too hard to find a correlation 
here. You don’t know these people, you 
don’t know what they intended. You try to 
compile statistics and correlate them to a 


result that amounts to nothing more than 
speculation. 9 9 


Marc Racicot, former governor of Montana 
and chairman of the Republican Party 


bringingresearchtolife 


The executive director of Glacier Symphony gestures 
broadly at the still snowcapped Canadian Rockies. “It 
has been three very happy years for me here, though 
not easy ones since I let corporate America intrude on 
our idyllic existence.” 

“You mean the MindWriter people?” prompts Jason 
Henry. “The ones who flew me up here? My clients and 
your benefactor?” 

“Please, don’t misunderstand,” says the executive 
director as she propels Jason across a manicured lawn 
toward the refreshment tent. “When I rented them a 
part of our compound for use in corporate education, 
they quite generously insisted that I avail myself of 
some of their training for midlevel managers.” 

“They 
attendance?” ventures Jason. “Tell me what you do 


said you were having trouble with 
heres 

“We offer one of the most outstanding summer 
music festivals in the country—maybe the continent. 
We present several concerts each week, all summer 
long, with evening performances on both Friday and 
Saturday. During the week, rehearsals are open to 
music patrons and students. And, of course, our skilled 
musicians enhance their own skills by networking with 
each other. 

“During the winter my artistic directors prepare 
the next summer’s program and hire the musicians, 
coordinating closely with me on the budget. This is 
quite complicated, because most of our musicians 
spend only two weeks with us. Fully 600 performing 
artists from many parts of the continent are part of this 
orchestra over the course of a summer festival. 

“Colleges in British Columbia send me _ their 
music scholarship students for summer employment 
as dishwashers, waiters, cleaners, and the like. It is 
a special opportunity for them, rubbing shoulders 


with their idols and learning to enhance their own 
performance skills in the process.” 

“So your problem is. . . ?” urges Jason again. 

“My problem is patronage, specifically the lack 
of commitment of the local residents of Glacier to 
consistently support their Glacier Symphony Festival. 
Do you realize how rare it is for a town this size to have 
more than 600 performing musicians in a summer? 
You would think the residents would be as ecstatic as 
our dishwashers!” 

“Do you know why they are less than supportive?” 
inquires Jason, glad they have finally arrived at 
the reason MindWriter had asked him to divert his 
homebound flight from San Francisco to British 
Columbia. 

“Well, some of the residents have communicated 
with us informally,” comments the director, somewhat 
hesitantly. 

“And they said... ?” urges Jason, more than a 
little impatiently, remembering why he so values his 
partner for usually handling this phase of exploratory 
research. 

“One commented: ‘I’ve never heard this music 
before—why can’t the performers play something [’ll 
recognize.’ Another, “Where were the video screens? 
And the special visual effects?’ And another: ‘Why 
would I want to spend more than an hour watching a 
stage full of people sitting in chairs?’” 

“Hold on,” says Jason, making a note in his 
smartphone. “I can see your orchestra is striking a sour 
note.” Jason smiles, chuckling at his own wit, while 
the director remains stoic. “MindWriter uses an 
extensive program for measuring customer satisfaction, 
andes 

“Ah, yes, measuring customer satisfaction,” 
interrupts the director, “second only to cash flow 
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for the MindWriter folks. The care and frequency 
with which they measure customer satisfaction in 
the MindWriter seminars here dumbfounds me. 
Throughout each seminar they host here, morning, 
afternoon, or evening, everyone breaks for coffee and 
is required to fill out a critique of the speaker. The 
results are tabulated by the time the last coffee cup 
has been collected, and the seminar leader has been 
given feedback. Is he or she presenting material too 
slowly or too quickly? Are there too many jokes or 
not enough? Are concrete examples being used often 
enough? Do the participants want a hard copy of the 
slides? They measure attitudinal data six times a day 
and even query you about the meals, including taste, 
appearance, cleanliness and speed, friendliness, and 
accuracy of service.” 

“Understandable,” observes Jason. “Your scholar- 
ship students have frequent contact with the residents, 
both here and in town, right? We might use them to 
collect some more formal data,” brainstorms Jason to 
himself. 

“Jason,” interjects the director, “were you ever a 
musician?” 

“No,” explains Jason, “my interests ran more toward 
statistics than Schubert.” 


bringingresearchtolifecont’d 


“Then you wouldn’t realize that while musicians 
could talk 
performing music—for hours with each other, once a 


about music—and the intricacies of 
resident exclaims little or no interest, our scholarship 
students would likely tune them out.” 

“Tt is just as well,’ comments Jason, now resigned to 
getting more involved in Glacier Symphony’s problem 
than he had first assumed would be necessary. “Untrained 
interviewers and observers can be highly unreliable and 
inaccurate in measuring and reporting behavior,” says 
Jason. “Have you tried a suggestion box?” 

“No, but I do send reminder postcards for each 
concert.” 


> 


“Not quite the same thing,” murmurs Jason as he 
hands the director his business card. “As a devotee 
of the MindWriter way, I’m sure you have a current 
satisfaction survey for concert goers in your files.” At her 
nod, Jason continues: “Send it to me. At MindWriter’s 
request and at its expense, I'll revise it for you. Il work 
out the collection and analysis details on my flight home 
and be in touch next week.” 

The director, smiling and shaking Jason’s hand, 
responds, “I’ll ask one of our scholarship students to 
drive you back to the community airport, then. You’re 
bound to have a lot in common.” 


In everyday usage, measurement occurs when an established index verifies the height, weight, or other 
feature of a physical object. How well you like a song, a painting, or the personality of a friend is 
also a measurement. To measure is to discover the extent, dimensions, quantity, or capacity of some- 
thing, especially by comparison with a standard. We measure casually in daily life, but in research the 
requirements are rigorous. 

Measurement in research consists of assigning numbers to empirical events, objects or properties, 
or activities in compliance with a set of rules. This definition implies that measurement is a three-part 
process: 


1. Selecting observable empirical events. 


2. Developing a set of mapping rules: a scheme for assigning numbers or symbols to represent 
aspects of the event being measured. 


3. Applying the mapping rule(s) to each observation of that event.’ 


You recall the term empirical. Researchers use an empirical approach to describe, explain, and make 
predictions by relying on information gained through observation. 
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Assume you are studying people who attend an auto show where prototypes for new models are on 
display. You are interested in learning the male-to-female ratio among attendees. You observe those 
who enter the show area. If a person is female, you record an F; if male, an M. Any other symbols such 
as 0 and | or # and % also may be used if you know what group the symbol identifies. Exhibit 11-1 uses 
this example to illustrate the three components. 


>Exhibit 11-1 Characteristics of Measurement 


Gender Attendees Styling Characteristics Attendees 
Sample 
Elements ABCODE ABCODE 
Empirical —| Gender Desirability 
Observations of auto styling 
Mapping Rule Assign Assign 
M if male 5 if very desirable 
F if female 4 if desirable 
3 if neither 
2 if undesirable 
1 if very undesirable 
Symbol (1 through 5) 


Attendees A, B, and C are male, and find the auto’s styling to be undesirable. 
Attendees D and E are female and find the auto’s styling desirable. 
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Researchers might also want to measure the styling desirability of a new concept car at this show. 
They interview a sample of visitors and assign, with a different mapping rule, their opinions to the fol- 
lowing scale: 


What is your opinion of the styling of the concept CS? 
Very desirable 5 4 3 2 1 Very undesirable 


All measurement theorists would call the rating scale in Exhibit 11-1 a form of measurement, but 
some would challenge whether classifying males and females is a form of measurement. Their 
argument is that measurement must involve quantification—that is, “the assignment of numbers to 
objects to represent amounts or degrees of a property possessed by all of the objects.”? This con- 
dition was met when measuring opinions of car styling. Our approach endorses the more general 
view that “numbers as symbols within a mapping rule” can reflect both qualitative and quantitative 
concepts. 

The goal of measurement—indeed, the goal of ‘assigning numbers to empirical events in compli- 
ance with a set of rules”—is to provide the highest-quality, lowest-error data for testing hypotheses, 
estimation or prediction, or description. Researchers deduce from a hypothesis that certain conditions 
should exist. Then they measure for these conditions in the real world. If found, the data lend support 
to the hypothesis; if not, researchers conclude the hypothesis is faulty. An important question at this 
point is, “Just what does one measure?” 

The object of measurement is a concept, the symbols we attach to bundles of meaning that we 
hold and share with others. We invent higher-level concepts—constructs—for specialized scien- 
tific explanatory purposes that are not directly observable and for thinking about and communicat- 
ing abstractions. Concepts and constructs are used at theoretical levels; variables are used at the 
empirical level. Variables accept numerals or values for the purpose of testing and measurement. 
Concepts, constructs, and variables may be defined descriptively or operationally. An operational 
definition defines a variable in terms of specific measurement and testing criteria. It must specify 
adequately the empirical information needed and how it will be collected. In addition, it must have 
the proper scope or fit for the research problem at hand. We review these terms with examples in 
Exhibit 11-2. 


What Is Measured? 


Variables being studied in research may be classified as objects or as properties. Objects include the 
concepts of ordinary experience, such as tangible items like furniture, laundry detergent, people, or au- 
tomobiles. Objects also include things that are not as concrete, such as genes, attitudes, and peer-group 
pressures. Properties are the characteristics of the object. A person’s physical properties may be stated 
in terms of weight, height, and posture, among others. Psychological properties include attitudes and 
intelligence. Social properties include leadership ability, class affiliation, and status. These and many 
other properties of an individual can be measured in a research study. 

In a literal sense, researchers do not measure either objects or properties. They measure indicants of 
the properties or indicants of the properties of objects. It is easy to observe that A is taller than B and 
that C participates more than D in a group process. Or suppose you are analyzing members of a sales 
force of several hundred people to learn what personal properties contribute to sales success. The prop- 
erties are age, years of experience, and number of calls made per week. The indicants in these cases are 
so accepted that one considers the properties to be observed directly. 

In contrast, it is not easy to measure properties of constructs like “lifestyles,” “opinion leader- 
ship,” “distribution channel structure,” and “persuasiveness.” Since each property cannot be measured 
directly, one must infer its presence or absence by observing some indicant or pointer measurement. 
When you begin to make such inferences, there is often disagreement about how to develop an opera- 
tional definition for each indicant. 

Not only is it a challenge to measure such constructs, but a study’s quality depends on what mea- 
sures are selected or developed and how they fit the circumstances. The nature of measurement scales, 
sources of error, and characteristics of sound measurement are considered next. 
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>Exhibit 11-2 Review of Key Terms 


Concept: a bundle of meanings or characteristics associated with certain events, objects, conditions, situations, or 
behaviors. 


Classifying and categorizing objects or events that have common characteristics beyond any single observation creates 
concepts. When you think of a spreadsheet or a warranty card, what comes to mind is not a single example but your 
collected memories of all spreadsheets and warranty cards from which you abstract a set of specific and definable 
characteristics. 


Variable: an event, act, characteristic, trait, or attribute that can be measured and to which we assign numerals or values; 
a synonym for the construct or the property being studied. 


The numerical value assigned to a variable is based on the variable’s properties. For example, some variables, said to be 
dichotomous, have only two values, reflecting the presence or absence of a property: employed-unemployed or male-female 
have two values, generally 0 and 1. Variables also take on values representing added categories, such as the demographic 
variables of race and religion. All such variables that produce data that fit into categories are discrete variables, since only 

certain values are possible. An automotive variable, for example, where “Chevrolet” is assigned a 5 and “Honda” is assigned a 

6 provides no option for a 5.5. Income, temperature, age, and a test score are examples of continuous variables. These variables 
may take on values within a given range or, in some cases, an infinite set. Your test score may range from O to 100, your age 
may be 23.5, and your present income could be $35,000. 


> Measurement Scales 


In measuring, one devises some mapping rule and then translates the observation of property in- 
dicants using this rule. For each concept or construct, several types of measurement are possible; 
the appropriate choice depends on what you assume about the mapping rules. Each one has its 
own set of underlying assumptions about how the numerical symbols correspond to real-world 
observations. 

Mapping rules have four assumptions: 


1. Numbers are used to classify, group, or sort responses. No order exists. 


2. Numbers are ordered. One number is greater than, less than, or equal to another number. 


3. Differences between numbers are ordered. The difference between any pair of numbers is 
greater than, less than, or equal to the difference between any other pair of numbers. 


4. The number series has a unique origin indicated by the number zero. This is an absolute and 
meaningful zero point. 
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>Exhibit 11-3 Measurement Scales 


Type of Scale 


Characteristics of Data Basic Empirical Operation Example 


Nominal 


Classification (mutually exclusive and Determination of equality Gender (male, female) 
collectively exhaustive categories), but 
no order, distance, or natural origin 


Interval 


Classification, order, and distance, but no Determination of equality of Temperature in degrees 
natural origin intervals or differences 


Combinations of these characteristics of classification, order, distance, and origin provide four 
widely used classifications of measurement scales:* (1) nominal, (2) ordinal, (3) interval, and (4) ratio. 
Let’s preview these measurement scales before we discuss their technical details. Suppose your profes- 
sor asks a student volunteer to taste-test six candy bars. The student begins by evaluating each on a 
chocolate—not chocolate scale; this is a nominal measurement. Then the student ranks the candy bars 
from best to worst; this is an ordinal measurement. Next, the student uses a 7-point scale that has equal 
distance between points to rate the candy bars with regard to some taste criterion (e.g., crunchiness); 
this is an interval measurement. Finally, the student considers another taste dimension and assigns 
100 points among the six candy bars; this is a ratio measurement. 

The characteristics of these measurement scales are summarized in Exhibit 11-3. Deciding which 
type of scale is appropriate for your research needs should be seen as a part of the research process, as 
shown in Exhibit 11-4. 


Nominal Scales 


In business research, nominal data are widely used. With nominal scales, you are collecting information 
on a variable that naturally or by design can be grouped into two or more categories that are mutually 
exclusive and collectively exhaustive. If data were collected from the symphony patrons at the Glacier 
compound, patrons could be classified by whether they had attended prior symphony performances or 
this was their first time. Every patron would fit into one of the two groups within the variable attendance. 

The counting of members in each group is the only 


possible arithmetic operation when a nominal scale is em- 
ployed. If we use numerical symbols within our mapping 
tule to identify categories, these numbers are recognized as 
labels only and have no quantitative value. For example, the 
number 13 on a license plate does not imply the number of 
traffic tickets the registered owner has received, or the num- 
ber of accidents the car has been involved in, or the number 
of state lines it has crossed, and not the level of driving skill 
of its owner; it is only a means of identification as it is as- 


signed to a particular vehicle. 

Nominal classifications may consist of any number of separate groups if the groups are mutually 
exclusive and collectively exhaustive. Thus, one might classify the students in a course according to 
their expressed religious preferences. Mapping rule A given in the table is not a sound nominal scale 
because its categories are not mutually exclusive or collectively exhaustive. Mapping rule B meets the 
minimum requirements; it covers all the major religions and offers an “other” option. Nominal scales 
are the least powerful of the four data types. They suggest no order or distance relationship and have 
no arithmetic origin. The scale wastes any information a sample element might share about varying 
degrees of the property being measured. 
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Religious Preferences 


Mapping Rule A Mapping Rule B 


>Exhibit 11-4 Moving from Investigative to Measurement Questions 


What Do | Plan to Do with the Data? 


Is what I need to know a... 


Exploration? 


Discovering of 
difference? 


Finding of 
relationships? 


Sete me What Scale Type Do | Need? How many 
expected to be groups will be 


normal? Nominal compared? 


What is my Are groups 


expected related or 
sample size? independent? 
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Since the only quantification is the number count of cases in each category (the frequency distribu- 
tion), the researcher is restricted to the use of the mode as the measure of central tendency.* The mode 
is the most frequently occurring value. You can conclude which category has the most members, 
but that is all. There is no generally used measure of dispersion for nominal scales. Dispersion des- 
cribes how scores cluster or scatter in a distribution. By cross-tabulating nominal variables with other 
variables, you can begin to discern patterns in data. 

Although nominal data are statistically weak, they are still useful. If no other scale can be used, 
one can almost always classify a set of properties into a set of equivalent classes. Nominal measures 
are especially valuable in exploratory work where the objective is to uncover relationships rather than 
secure precise measurements. This type of scale is also widely used in survey and other research when 
data are classified by major subgroups of the population. Classifications such as respondents’ marital 
status, gender, political orientation, and exposure to a certain experience provide insight into important 
demographic data patterns. 

Jason visited Glacier because of his familiarity with MindWriter’s extensive research into customer 
satisfaction. His visit revealed Glacier’s need for some exploratory nominal data on symphony patrons. 
Patrons could be divided into groups—based on their appreciation of the conductor (favorable, unfa- 
vorable), on their attitude toward facilities (suitable, not suitable), on their perception of the program 
(clichéd, virtuoso), on their level of symphony support (financial support, no financial support)—and 
then analyzed. 


Ordinal Scales 


Ordinal scales include the characteristics of the nominal scale plus an indication of order. Ordinal data 
require conformity to a logical postulate, which states: If ais greater than b and b is greater than c, then 
ais greater than c.° The use of an ordinal scale implies a statement of “greater than” or “less than” (an 
equality statement is also acceptable) without stating how much greater or less. While ordinal measure- 
ment speaks of greater-than and less-than measurements, other descriptors may be used—“superior 
to,” “happier than,” “poorer than,” or “important than.” Like a rubber yardstick, an ordinal scale can 
stretch varying amounts at different places along its length. Thus, the real difference between ranks | 
and 2 on a satisfaction scale may be more or less than the difference between ranks 2 and 3. An ordi- 
nal concept can be extended beyond the three cases used in the simple illustration of a > b > c. Any 
number of cases can be ranked. 

Another extension of the ordinal concept occurs when there is more than one property of interest. 
We may ask a taster to rank varieties of carbonated soft drinks by flavor, color, carbonation, and a com- 
bination of these characteristics. We can secure the combined ranking either by asking the respondent 
to base his or her ranking on the combination of properties or by constructing a combination ranking of 
the individual rankings on each property. 

Examples of ordinal data include attitude and preference scales. (In the next chapter, we provide 
detailed examples of attitude scales.) Because the numbers used with ordinal scales have only a rank 
meaning, the appropriate measure of central tendency is the median. The median is the midpoint of a 
distribution. A percentile or quartile reveals the dispersion. 

Correlational analysis of ordinal data is restricted to various ordinal techniques. Measures of sta- 
tistical significance are technically confined to a body of statistics known as nonparametric methods, 
synonymous with distribution-free statistics.° 

Researchers differ about whether more powerful tests are appropriate for analyzing ordinal mea- 
sures. Because nonparametric tests are abundant, simple to calculate, have good statistical power,’ 
and do not require that the researcher accept the assumptions of parametric testing, we advise their use 
with nominal and ordinal data. It is understandable, however, that because parametric tests (such as the 
t-test or analysis of variance) are versatile, accepted, and understood, they will be used with ordinal 
data when researchers can demonstrate that those data approach the assumptions necessary for interval 
level analysis. 
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Interval Scales 


Interval scales have the power of nominal and ordinal data plus one additional strength: They 
incorporate the concept of equality of interval (the scaled distance between | and 2 equals the dis- 
tance between 2 and 3). Calendar time is such a scale. For example, the elapsed time between 3 and 
6 a.m. equals the time between 4 and 7 a.m. One cannot say, however, that 6 a.m. is twice as late as 
3 a.m., because “zero time” is an arbitrary zero point. Centigrade and Fahrenheit temperature scales 
are other examples of classical interval scales. Both have an arbitrarily determined zero point, not 
a unique origin. 

Researchers treat many attitude scales as interval, as we illustrate in the next chapter. When a scale 
is interval and the data are relatively symmetric with one mode, you use the arithmetic mean as the 
measure of central tendency. You can compute the average time of a TV promotional message or the 
average attitude value for different age groups in an insurance benefits study. The standard deviation is 
the measure of dispersion. The product-moment correlation, f-tests, F-tests, and other parametric tests 
are the statistical procedures of choice for interval data.* 

When the distribution of scores computed from interval data leans in one direction or the other 
(skewed right or left), we often use the median as the measure of central tendency and the inter- 
quartile range as the measure of dispersion. The reasons for this are discussed in the end-of-chapter 
Appendix 15a. 


Ratio Scales 


Ratio scales incorporate all of the powers of the pre- 
vious scales plus the provision for absolute zero or 
origin. Ratio data represent the actual amounts of a 
variable. Measures of physical dimensions such as 
weight, height, distance, and area are examples. In the 
behavioral sciences, few situations satisfy the require- 
ments of the ratio scale—the area of psychophysics of- 
fering some exceptions. In business research, we find 
ratio scales in many areas. There are money values, Gophers burrow through life without seeing the havoc they create. 
population counts, distances, return rates, productivity 
rates, and amounts of time (e.g., elapsed time in sec- ST svete 

onds before a customer service representative answers But you can. With proven business intelligence and analytic software from SAS. 
a phone inquiry). www.sas.com/gophers 

Swatch’s BeatTime—a proposed standard global 
time introduced at the 2000 Olympics that may gain 
favor as more of us participate in cross-time-zone 
chats (Internet or otherwise)—is a ratio scale. It offers 
a standard time with its origin at 0 beats (12 midnight 
in Biel, Switzerland, at the new Biel Meridian time- 
line). A day is composed of 1,000 beats, with a “beat” 
worth | minute, 26.4 seconds.’ 

With the Glacier project, Jason could measure a 
customer’s age, the number of years he or she has at- 
tended, and the number of times a selection has been 
performed in the Glacier summer festival. These mea- 
sures all generate ratio data. For practical purposes, 
however, the analyst would use the same statistical 
techniques as with interval data. In measurement, a researcher needs to know precisely what is being 

All statistical techniques mentioned up to this measured. As this SAS ad indicates, not having a good view of the big 
point are usable with ratio scales. Other manipulations __ picture is unacceptable. www.sas.com 
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Measurement of TiVo Households: Skipped Ads vs. Most Watched 


Doesn't it seem odd that some of the most popular shows on 
TV have the least watched ads? From TiVo research we dis- 
cover that, “nearly all of the television shows that won 2009 
Emmys showed higher levels of ad-skipping than the averages 
for their respective genres.”* Sitcoms had a 66 percent level of 
ad-skipping in contrast to the 88 percent who fast-forwarded 
through “MadMen’” ads and the 73 percent of the audiences who 
skipped over ads for all TV dramas.° According to Todd Juenger, 
TiVo’s vice president for audience research and measurement, 
people who watch hit shows are more likely to skip ads because 
they are more involved in the show than other viewers.° 
Madison Avenue media planners have long counseled com- 
panies to avoid buying ads in the fourth quarter of the Super Bowl, 
even understanding that part of the appeal of the Super Bowl is 
watching the ads. However, such advice is now being reassessed 


ast quarter of Super Bowl XLIII had strong viewer numbers. Two 
of the most-watched ads on TiVo DVRs were the final ads in the 
game: Bud Light Lime beer and GoDaddy.com, the website reg- 
istration firm. These ads were broadcast after the Arizona Cardi- 
nals touchdown and before the Pittsburgh Steelers reclaimed the 
lead and won. “There are two reasons a commercial gets a high 
rating [in a TiVo household],” Mr. Juenger said. “Either [the show] 
is rewound often and watched repeatedly or [the ad] happens to 
be in the middle of [compelling programming that is watched over 
and over].° 

Those who sat on the edge of their seats for the end of Super 
Bowl XLIII helped rank it as the second-most-watched Super 
Bowl. For those rewatching the end of the fourth quarter, sug- 
gest some hypotheses that explain ad-skipping. What measures 
would you use to test your hypotheses? 


after the second championship in consecutive years was won in 


the final moments of play.? NBC’s broadcast of commercials in the www.tivo.com 


carried out with real numbers may be done with ratio-scale values. Thus, multiplication and divi- 
sion can be used with this scale but not with the others mentioned. Geometric and harmonic means 
are measures of central tendency, and coefficients of variation may also be calculated for describing 
variability. 

Researchers often encounter the problem of evaluating variables that have been measured on dif- 
ferent scales. For example, the choice to purchase a product by a consumer is a nominal variable, and 
cost is a ratio variable. Certain statistical techniques require that the measurement levels be the same. 
Since the nominal variable does not have the characteristics of order, distance, or point of origin, we 
cannot create them artificially after the fact. The ratio-based salary variable, on the other hand, can be 
reduced. Rescaling product cost into categories (e.g., high, medium, low) simplifies the comparison. 
This example may be extended to other measurement situations—that is, converting or rescaling a 
variable involves reducing the measure from the more powerful and robust level to a lesser one.'° The 
loss of measurement power with this decision means that lesser-powered statistics are then used in 
data analysis, but fewer assumptions for their proper use are required. 

In summary, higher levels of measurement generally yield more information. Because of the mea- 
surement precision at higher levels, more powerful and sensitive statistical procedures can be used. As 
we saw with the candy bar example, when one moves from a higher measurement level to a lower one, 
there is always a loss of information. Finally, when we collect information at higher levels, we can 
always convert, rescale, or reduce the data to arrive at a lower level. 


> Sources of Measurement Differences 


The ideal study should be designed and controlled for precise and unambiguous measurement of the 
variables. Since complete control is unattainable, error does occur. Much error is systematic (results 
from a bias), while the remainder is random (occurs erratically). One authority has pointed out several 
sources from which measured differences can come."! 
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Talent Analytics: The Frontier of HR Research 


The biggest investment for most firms is their people. These firms Elements of a Talent Audit 
would love to Know whether they are attracting, recruiting, and de- 
veloping the best people given their type of business and the mar- 
ket for talent for a given position. According to research by Bersin 
& Associates, “Initially, many [HR] assessments were used primarily 
for pre-hire selection; today, assessment providers offer strategic 
resources for global organizations that want to build their overall 
knowledge of their employees beyond hiring—for development, 
promotion, internal mobility and succession planning.” 

People intelligence, as defined by Bersin is “the collected body 
of information on an individual over the course of that individual’s 
affiliation with an organization—from pre-application through 
emeritus stages. It is comprised of data that, when analyzed, is 
useful both to the individual and to the organization as a whole for 
strategic talent and business decision making and performance 
improvement.” Such people intelligence is gaining increasing in- 
terest as technology makes it possible to connect business out- 
comes more closely with the people who deliver them. 

SHL offers a research process called Talent Analytics for this 
purpose. In its 2012 Business Outcomes Study Report, cover- 
ing more than 66 business outcome studies, SHL demonstrated 
that great assessment tools and competitive benchmarking can 
attract better talent (people with the skills and attributes to better 
fit an organization and provide better bottom-line results), retain 
talent, enhance productivity, and reduce the negative effects 
of a rejected hire. Clients—like Time Warner, GlaxoSmithKline, 
Kellogg’s, and Hilton Hotels—choose from more than 1,000 
off-the-shelf, prepackaged assessment solutions delivered via a 
secure, user-friendly online platform. 

The results of employing these measurement tools are im- 
pressive. An automotive parts retailer discovered sales associates 
who earned higher scores on an assessment designed to predict 
sales potential sold 21 percent more over their first three months 
on the job. An insurance company used talent analytics to reduce 
employee turnover by 40 percent. Restaurant managers earning www.shl.com; www.bersin.com 


Results 
(what do employees deliver) 


Actions 
(what can employees 

do; competencies, 
skills) 


high scores on the talent analytic metrics of problem solving and 
organizational fit were twice as likely to be great performers. A 
car manufacturer using talent analytics measuring dependability 
and safety demonstrated that high scorers were twice as likely to 
be rated by superiors as having high potential for advancement. 

Bersin & Associates is a consultancy designed to improve a 
firm’s enterprise learning and talent management strategies, pro- 
cesses, and systems. SHL is the result of a merger between SHL 
and PreVisor, two talent assessment pioneers. As the world’s 
largest talent assessment firm, SHL performs more than 25 mil- 
lion scientifically proven selection and development assessments 
each year in more than 30 languages. You can learn more about 
talent analytics by visiting its website. 


Assume you are conducting an ex post facto study of corporate citizenship of a multinational manu- 
facturer. The company produces family, personal, and household care products. The participants are 
residents of a major city. The study concerns the Prince Corporation, a large manufacturer with its 
headquarters and several major facilities located in the city. The objective of the study is to discover the 
public’s opinions about the company’s approach to health, social welfare, and the environment. You 
also want to know the origin of any generally held adverse opinions. 

Ideally, any variation of scores among the respondents would reflect true differences in their opin- 
ions about the company. Attitudes toward the firm as an employer, as an ecologically sensitive organi- 
zation, Or as a progressive corporate citizen would be accurately expressed. However, four major error 
sources may contaminate the results: (1) the respondent, (2) the situation, (3) the measurer, and (4) the 
data collection instrument. 
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Error Sources 
The Respondent 


Opinion differences that affect measurement come from relatively stable characteristics of the respon- 
dent. Typical of these are employee status, ethnic group membership, social class, and nearness to 
manufacturing facilities. The skilled researcher will anticipate many of these dimensions, adjusting 
the design to eliminate, neutralize, or otherwise deal with them. However, even the skilled researcher 
may not be as aware of less obvious dimensions. The latter variety might be a traumatic experience a 
given participant had with the Prince Corporation, its programs, or its employees. Respondents may be 
reluctant to express strong negative (or positive) feelings, may purposefully express attitudes that they 
perceive as different from those of others, or may have little knowledge about Prince but be reluctant 
to admit ignorance. This reluctance to admit ignorance of a topic can lead to an interview consisting of 
“guesses” or assumptions, which, in turn, create erroneous data. 

Respondents may also suffer from temporary factors like fatigue, boredom, anxiety, hunger, impa- 
tience, or general variations in mood or other distractions; these limit the ability to respond accurately 
and fully. Designing measurement scales that engage the participant for the duration of the measurement 
is crucial. 


Situational Factors 


Any condition that places a strain on the interview or measurement session can have serious effects on 
the interviewer-respondent rapport. If another person is present, that person can distort responses by 
joining in, by distracting, or by merely being there. If the respondents believe anonymity is not ensured, 
they may be reluctant to express certain feelings. Curbside or intercept interviews are unlikely to elicit 
elaborate responses, while in-home interviews more often do. 


The Measurer 


The interviewer can distort responses by rewording, paraphrasing, or reordering questions. Stereotypes in ap- 
pearance and action introduce bias. Inflections of voice and conscious or unconscious prompting with smiles, 
nods, and so forth, may encourage or discourage certain replies. Careless mechanical processing—checking 
of the wrong response or failure to record full replies—will obviously distort findings. In the data analysis 
stage, incorrect coding, careless tabulation, and faulty statistical calculation may introduce further errors. 


The Instrument 


A defective instrument can cause distortion in two major ways. First, it can be too confusing and am- 
biguous. The use of complex words and syntax beyond participant comprehension is typical. Leading 
questions, ambiguous meanings, mechanical defects (inadequate space for replies, response-choice 
omissions, and poor printing), and multiple questions suggest the range of problems. Many of these 
problems are the direct result of operational definitions that are insufficient, resulting in an inappropri- 
ate scale being chosen or developed. 

A more elusive type of instrument deficiency is poor selection from the universe of content items. 
Seldom does the instrument explore all the potentially important issues. The Prince Corporation study 
might treat company image in areas of employment and ecology but omit the company management’s 
civic leadership, its support of local education programs, its philanthropy, or its position on minority 
issues. Even if the general issues are studied, the questions may not cover enough aspects of each area 
of concern. Although we might study the Prince Corporation’s image as an employer in terms of salary 
and wage scales, promotion opportunities, and work stability, perhaps such topics as working condi- 
tions, company management relations with organized labor, and retirement and other benefit programs 
should also be included. 
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> The Characteristics of Good Measurement 


What are the characteristics of a good measurement tool? An intuitive answer to this question is that 
the tool should be an accurate counter or indicator of what we are interested in measuring. In addition, 
it should be easy and efficient to use. There are three major criteria for evaluating a measurement tool: 
validity, reliability, and practicality. 


e Validity is the extent to which a test measures what we actually wish to measure. 
¢ Reliability has to do with the accuracy and precision of a measurement procedure. 


¢ Practicality is concerned with a wide range of factors of economy, convenience, 
and interpretability.’ 


In the following sections, we discuss the nature of these qualities and how researchers can achieve them 
in their measurement procedures. 


Validity 


Many forms of validity are mentioned in the research literature, and the number grows as we expand 
the concern for more scientific measurement. This text features two major forms: external and inter- 
nal validity.'’ The external validity of research findings is the data’s ability to be generalized across 
persons, settings, and times; we discussed this in reference to experimentation in Chapter 9, and more 
will be said in Chapter 14 on sampling.’ In this chapter, we discuss only internal validity. Internal 
validity is further limited in this discussion to the ability of a research instrument to measure what it is 
purported to measure. Does the instrument really measure what its designer claims it does? 

One widely accepted classification of validity consists of three major forms: (1) content validity, 
(2) criterion-related validity, and (3) construct validity (see Exhibit 11-5).'° 


Content Validity 


The content validity of a measuring instrument is the extent to which it provides adequate coverage 
of the investigative questions guiding the study. If the instrument contains a representative sample 
of the universe of subject matter of interest, then content validity is good. To evaluate the content 


>Exhibit 11-5 Summary of Validity Estimates 


What Is Measured Methods 


Content Degree to which the content of the items 
adequately represents the universe of 


all relevant items under study. 


e Judgmental 
e Panel evaluation with content validity 
ratio 


Construct Answers the question, “What accounts 
for the variance in the measure?”; 
attempts to identify the underlying 
construct(s) being measured and 
determine how well the test repre- 


sents it (them). 


Judgmental 

¢ Correlation of proposed test with 
established one 

e Convergent-discriminant techniques 

e Factor analysis 

e Multitrait-multimethod analysis 
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validity of an instrument, one must first agree on what elements constitute adequate coverage. In the 
Prince Corporation study, we must decide what knowledge and attitudes are relevant to the measure- 
ment of corporate public image and then decide which forms of these opinions are relevant positions 
on these topics. In the Glacier study, Jason must first determine what factors are influencing customer 
satisfaction before determining if published indexes can be of value. If the data collection instrument 
adequately covers the topics that have been defined as the relevant dimensions, we conclude the instru- 
ment has good content validity. 

A determination of content validity involves judgment. First, the designer may determine it through 
a careful definition of the topic, the items to be scaled, and the scales to be used. This logical process is 
often intuitive and unique to each research designer. 

A second way is to use a panel of persons to judge how well the instrument meets the standards. The 
panel independently assesses the test items for an instrument as essential, useful but not essential, or 
not necessary. “Essential” responses on each item from each panelist are evaluated by a content valid- 
ity ratio, and those meeting a statistical significance value are retained. In both informal judgments and 
this systematic process, “content validity is primarily concerned with inferences about test construction 
rather than inferences about test scores.”’'® 

It is important not to define content too narrowly. If you were to secure only superficial expressions 
of opinion in the Prince Corporation attitude survey, it would probably not have adequate content 
coverage. The research should delve into the processes by which these attitudes came about. How did 
the respondents come to feel as they do, and what is the intensity of feeling? The same would be true 
of MindWriter’s evaluation of service quality and satisfaction. It is not enough to know a customer 
is dissatisfied. The manager charged with enhancing or correcting the program needs to know what 
processes, employees, parts, and time sequences within the CompleteCare program have led to that 
dissatisfaction. 


Criterion-Related Validity 


Criterion-related validity reflects the success of measures used for prediction or estimation. You 
may want to predict an outcome or estimate the existence of a current behavior or time perspective. An 
attitude scale that correctly forecasts the outcome of a purchase decision has predictive validity. An 
observational method that correctly categorizes families by current income class has concurrent valid- 
ity. Although these examples appear to have simple and unambiguous validity criteria, there are dif- 
ficulties in estimating validity. Consider the problem of estimating family income. There is a knowable 
true income for every family, but we may find the figure difficult to secure. Thus, while the criterion is 
conceptually clear, it may be unavailable. 

A researcher may want to develop a preemployment test that will predict sales success. There may 
be several possible criteria, none of which individually tells the full story. Total sales per salesperson 
may not adequately reflect territory market potential, competitive conditions, or the different profit- 
ability rates of various products. One might rely on the sales manager’s overall evaluation, but how 
unbiased and accurate are such impressions? The researcher must ensure that the validity criterion 
used is itself “valid.” Any criterion measure must be judged in terms of four qualities: (1) relevance, 
(2) freedom from bias, (3) reliability, and (4) availability.'’ 

A criterion is relevant if it is defined and scored in the terms we judge to be the proper measures of 
salesperson success. If you believe sales success is adequately measured by dollar sales volume achieved 
per year, then it is the relevant criterion. If you believe success should include a high level of penetration 
of large accounts, then sales volume alone is not fully relevant. In making this decision, you must rely on 
your judgment in deciding what partial criteria are appropriate indicants of salesperson success. 

Freedom from bias is attained when the criterion gives each salesperson an equal opportunity to 
score well. The sales criterion would be biased if it did not show adjustments for differences in territory 
potential and competitive conditions. 

A reliable criterion is stable or reproducible. An erratic criterion (using monthly sales, which are 
highly variable from month to month) can hardly be considered a reliable standard by which to judge 
performance on a sales employment test. Finally, the information specified by the criterion must be 
available. If it is not available, how much will it cost and how difficult will it be to secure? The amount 
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of money and effort that should be spent on development of a criterion depends on the importance of 
the problem for which the test is used. 

Once there are test and criterion scores, they must be compared in some way. The usual approach 
is to correlate them. For example, you might correlate test scores of 40 new salespeople with first-year 
sales achievements adjusted to reflect differences in territorial selling conditions. 


Construct Validity 


In attempting to evaluate construct validity, we consider both the theory and the measuring instrument 
being used. If we were interested in measuring the effect of trust in cross-functional teams, the way in 
which “trust” was operationally defined would have to correspond to an empirically grounded theory. 
If a known measure of trust was available, we might correlate the results obtained using this measure 
with those derived from our new instrument. Such an approach would provide us with preliminary in- 
dications of convergent validity (the degree to which scores on one scale correlate with scores on other 
scales designed to assess the same construct). If Jason were to develop a customer satisfaction index for 
Glacier and, when compared, the results revealed the same indications as a predeveloped, established 
index, Jason’s instrument would have convergent validity. Similarly, if Jason developed an instrument 
to measure satisfaction with the CompleteCare program and the derived measure could be confirmed 
with a standardized customer satisfaction measure, convergent validity would exist. 

Returning to our preceding example, another method of validating the trust construct would be 
to separate it from other constructs in the theory or related theories. To the extent that trust could be 
separated from bonding, reciprocity, and empathy, we would have completed the first steps toward 
discriminant validity (the degree to which scores on a scale do not correlate with scores from scales 
designed to measure different constructs). 

We discuss the three forms of validity separately, but they are interrelated, both theoretically and 
operationally. Predictive validity is important for a test designed to predict product success. In devel- 
oping such a test, you would probably first list the factors (constructs) that provide the basis for useful 
prediction. For example, you would advance a theory about the variables in product success—an area 
for construct validity. Finally, in developing the specific items for inclusion in the success prediction 
test, you would be concerned with how well the specific items sample the full range of each construct 
(a matter of content validity). Looking at Exhibit 11-6, we can better understand the concepts of valid- 
ity and reliability by using an archer’s bow and target as an analogy. 


>Exhibit 11-6 Understanding Validity and Reliability 


Validity Let’s use an archer’s bow and target as an analogy. 
High Low High reliability means that repeated arrows shot 
from the same bow would hit the target in 
essentially the same place—although not 
necessarily the intended place (first row of the 
graphic). If we had a bow with high validity as well, 
then every arrow would hit the bull’s-eye (upper left 
panel). If reliability is low or decreases for some 
reason, arrows would be more scattered (lacking 
similarity or closeness, like those shown in the 
second row). 

High validity means that the bow would shoot 
true every time. It would not pull to the right or send 
an arrow careening into the woods. Arrows shot 
from a high-validity bow will be clustered around a 
central point (the bull’s eye), even when they are 
dispersed by reduced reliability (ower left graphic). 
Low-validity shots are distorted, rather like a flawed 
bow pulling our arrows to the right (second column 
of the graphic). We wouldn’t hit the bull’s-eye we were aiming at because the low-validity bow—like the 
flawed data collection instrtument—would not perform as planned. When low validity is compounded by low 
reliability, the pattern of arrows is not only off the bull’s-eye but also dispersed (lower right graphic). 


High 


Reliability 


Low 
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>Exhibit 11-7 Summary of Reliability Estimates 


Coefficient What Is Measured Methods 


Test-Retest Stability Reliability of a test or instrument inferred from examinee Correlation 
scores; same test is administered 
twice to same subjects over an interval of less than six 


months. 


Specialized correlational 
formulas 


Split-Half, KR20, 
Cronbach’s Alpha 


Internal 
consistency 


Degree to which instrument items are homogeneous and re- 
flect the same underlying construct(s). 


Reliability 


A measure is reliable to the degree that it supplies consistent results. Reliability is a necessary con- 
tributor to validity but is not a sufficient condition for validity. The relationship between reliability and 
validity can be simply illustrated with the use of a bathroom scale. If the scale measures your weight 
correctly (using a concurrent criterion such as a scale known to be accurate), then it is both reliable 
and valid. If it consistently overweighs you by six pounds, then the scale is reliable but not valid. If the 
scale measures erratically from time to time, then it is not reliable and therefore cannot be valid. So 
if a measurement is not valid, it hardly matters if it is reliable—because it does not measure what the 
designer needs to measure in order to solve the research problem. In this context, reliability is not as 
valuable as validity, but it is much easier to assess. 

Reliability is concerned with estimates of the degree to which a measurement is free of random or 
unstable error. Reliable instruments can be used with confidence that transient and situational factors 
are not interfering. Reliable instruments are robust; they work well at different times under differ- 
ent conditions. This distinction of time and condition is the basis for frequently used perspectives on 
reliability—stability, equivalence, and internal consistency (see Exhibit 11-7). 


Stability 


A measure is said to possess stability if you can secure consistent results with repeated measurements 
of the same person with the same instrument. An observation procedure is stable if it gives the same 
reading on a particular person when repeated one or more times. It is often possible to repeat obser- 
vations on a subject and to compare them for consistency. When there is much time between mea- 
surements, there is a chance for situational factors to change, thereby affecting the observations. The 
change would appear incorrectly as a drop in the reliability of the measurement process. 

Stability measurement in survey situations is more difficult and less easily executed than in obser- 
vational studies. Although you can observe a certain action repeatedly, you usually can resurvey only 
once. This leads to a test-retest arrangement—with comparisons between the two tests to learn how 
reliable they are. Some of the difficulties that can occur in the test-retest methodology and cause a 
downward bias in stability include: 


° Time delay between measurements—leads to situational factor changes (also a problem in obser- 
vation studies). 

° Insufficient time between measurements—permits the respondent to remember previous answers 
and repeat them, resulting in biased reliability indicators. 

° Respondent’s discernment of a study’s disguised purpose—may introduce bias if the respondent 
holds opinions related to the purpose but not assessed with current measurement questions. 

° Topic sensitivity—occurs when the respondent seeks to learn more about the topic or form new 
and different opinions before the retest. 
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A suggested remedy is to extend the interval between test and retest (from two weeks to a 
month). While this may help, the researcher must be alert to the chance that an outside factor will 
contaminate the measurement and distort the stability score. Consequently, stability measure- 
ment through the test-retest approach has limited applications. More interest has centered on 
equivalence. 


Equivalence 


A second perspective on reliability considers how much error may be introduced by different inves- 
tigators (in observation) or different samples of items being studied (in questioning or scales). Thus, 
while stability is concerned with personal and situational fluctuations from one time to another, 
equivalence is concerned with variations at one point in time among observers and samples of 
items. A good way to test for the equivalence of measurements by different observers is to compare 
their scoring of the same event. An example of this is the scoring of Olympic figure skaters by a 
panel of judges. 

In studies where a consensus among experts or observers is required, the similarity of the judges’ 
perceptions is sometimes questioned. How does a panel of supervisors render a judgment on merit 
raises, a new product’s packaging, or future business trends? Interrater reliability may be used in these 
cases to correlate the observations or scores of the judges and render an index of how consistent their 
ratings are. In Olympic figure skating, a judge’s relative positioning of skaters (determined by estab- 
lishing a rank order for each judge and comparing each judge’s ordering for all skaters) is a means of 
measuring equivalence. 

The major interest with equivalence is typically not how respondents differ from item to item but 
how well a given set of items will categorize individuals. There may be many differences in response 
between two samples of items, but if a person is classified the same way by each test, then the tests 
have good equivalence. 

One tests for item sample equivalence by using alternative or parallel forms of the same test ad- 
ministered to the same persons simultaneously. The results of the two tests are then correlated. Under 
this condition, the length of the testing process is likely to affect the subjects’ responses through fa- 
tigue, and the inferred reliability of the parallel form will be reduced accordingly. Some measurement 
theorists recommend an interval between the two tests to compensate for this problem. This approach, 
called delayed equivalent forms, is a composite of test-retest and the equivalence method. As in test- 
retest, one would administer form X followed by form Y to half the examinees and form Y followed by 
form X to the other half to prevent “order-of-presentation” effects." 

The researcher can include only a limited number of measurement questions in an instrument. 
This limitation implies that a sample of measurement questions from a content domain has been 
chosen and another sample producing a similar number will need to be drawn for the second instru- 
ment. It is frequently difficult to create this second set. Yet if the pool is initially large enough, 
the items may be randomly selected for each instrument. Even with more sophisticated procedures 
used by publishers of standardized tests, it is rare to find fully equivalent and interchangeable 
questions.!” 


Internal Consistency 


A third approach to reliability uses only one administration of an instrument or test to assess the 
internal consistency or homogeneity among the items. The split-half technique can be used when 
the measuring tool has many similar questions or statements to which the participant can respond. The 
instrument is administered and the results are separated by item into even and odd numbers or into ran- 
domly selected halves. When the two halves are correlated, if the results of the correlation are high, the 
instrument is said to have high reliability in an internal consistency sense. The high correlation tells us 
there is similarity (or homogeneity) among the items. The potential for incorrect inferences about high 
internal consistency exists when the test contains many items—which inflates the correlation index. 

The Spearman-Brown correction formula is used to adjust for the effect of test length and to esti- 
mate reliability of the whole test.”° 
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Practicality 


The scientific requirements of a project call for the measurement process to be reliable and valid, while 
the operational requirements call for it to be practical. Practicality has been defined as economy, con- 
venience, and interpretability.’ Although this definition refers to the development of educational and 
psychological tests, it is meaningful for business measurements as well. 


Economy 


Some trade-off usually occurs between the ideal research project and the budget. Data are not free, 
and instrument length is one area where economic pressures dominate. More items give more reli- 
ability, but in the interest of limiting the interview or observation time (and therefore costs), we 
hold down the number of measurement questions. The choice of data collection method is also often 
dictated by economic factors. The rising cost of personal interviewing first led to an increased use of 
telephone surveys and subsequently to the current rise in Internet surveys. In standardized tests, the 
cost of test materials alone can be such a significant expense that it encourages multiple reuse. Add 
to this the need for fast and economical scoring, and we see why computer scoring and scanning are 
attractive. 


Convenience 


A measuring device passes the convenience test if it is easy to administer. A questionnaire or a 
measurement scale with a set of detailed but clear instructions, with examples, is easier to com- 
plete correctly than one that lacks these features. In a well-prepared study, it is not uncommon 
for the interviewer instructions to be several times longer than the interview questions. Naturally, 
the more complex the concepts and constructs, the greater is the need for clear and complete in- 
structions. We can also make the instrument easier to administer by giving close attention to its 
design and layout. Although reliability and validity dominate our choices in design of scales here 
and later in Chapter 12, administrative difficulty should play some role. A long completion time, 
complex instructions, participant’s perceived difficulty with the survey, and their rated enjoyment 
of the process also influence design. Layout issues include crowding of material, poor reproduc- 
tions of illustrations, and the carryover of items from one page to the next or the need to scroll the 
screen when taking a Web survey. Both design and layout issues make completion of the instru- 
ment more difficult. 


Interpretability 


This aspect of practicality is relevant when persons other than the test designers must interpret the 
results. It is usually, but not exclusively, an issue with standardized tests. In such cases, the designer 
of the data collection instrument provides several key pieces of information to make interpretation 
possible: 


e A statement of the functions the test was designed to measure and the procedures by which it 
was developed. 

° Detailed instructions for administration. 

¢ Scoring keys and instructions. 

¢ Norms for appropriate reference groups. 

e Evidence about reliability. 

e Evidence regarding the intercorrelations of subscores. 

e Evidence regarding the relationship of the test to other measures. 


e Guides for test use. 
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Although people measure things casually in daily life, re- 
search measurement is more precise and controlled. In 
measurement, one settles for measuring properties of the 
objects rather than the objects themselves. An event is 
measured in terms of its duration. What happened during 
it, who was involved, where it occurred, and so forth, are all 
properties of the event. To be more precise, what are mea- 
sured are indicants of the properties. Thus, for duration, one 
measures the number of hours and minutes recorded. For 
what happened, one uses some system to classify types of 
activities that occurred. Measurement typically uses some 
sort of scale to classify or quantify the data collected. 


Ds) 


There are four scale types. In increasing order of power, they 
are nominal, ordinal, interval, and ratio. Nominal scales classify 
without indicating order, distance, or unique origin. Ordinal 
data show magnitude relationships of more than and less than 
but have no distance or unique origin. Interval scales have 
both order and distance but no unique origin. Ratio scales 
possess classification, order, distance, and unique origin. 


wo 


Instruments may yield incorrect readings of an indicant for 
many reasons. These may be classified according to error 


sources: (a) the respondent or participant, (b) situational 
factors, (c) the measurer, and (d) the instrument. 


4 Sound measurement must meet the tests of validity, reli- 


ability, and practicality. Validity reveals the degree to which 
an instrument measures what it is supposed to measure 

to assist the researcher in solving the research problem. 
Three forms of validity are used to evaluate measurement 
scales. Content validity exists to the degree that a measure 
provides an adequate reflection of the topic under study. Its 
determination is primarily judgmental and intuitive. Criterion- 
related validity relates to our ability to predict some outcome 
or estimate the existence of some current condition. Con- 
struct validity is the most complex and abstract. A measure 
has construct validity to the degree that it conforms to 
predicted correlations of other theoretical propositions. 

A measure is reliable if it provides consistent results. 
Reliability is a partial contributor to validity, but a measure- 
ment tool may be reliable without being valid. Three forms of 
reliability are stability, equivalence, and internal consistency. 
A measure has practical value for the research if it is eco- 
nomical, convenient, and interpretable. 
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pdiscussiong ection: 


Terms in Review 
1 What can we measure about the four objects listed below? 
Be as specific as possible. 


a Laundry detergent 
b Employees 

c Factory output 

d Job satisfaction 


2 What are the essential differences among nominal, 
ordinal, interval, and ratio scales? How do these 
differences affect the statistical analysis techniques 
we can use? 


3 What are the four major sources of measurement error? 
Illustrate by example how each of these might affect mea- 
surement results in a face-to-face interview situation. 


4 Do you agree or disagree with the following statements? 
Explain. 


a Validity is more critical to measurement than reliability. 


b Content validity is the most difficult type of validity to 
determine. 


c Avalid measurement is reliable, but a reliable measure- 
ment may not be valid. 


d Stability and equivalence are essentially the same thing. 
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Making Research Decisions 
5 You have data from a corporation on the annual salary of 


each of its 200 employees. 


a Illustrate how the data can be presented as ratio, inter- 
val, ordinal, and nominal data. 


b Describe the successive loss of information as the pre- 
sentation changes from ratio to nominal. 


Below are listed some objects of varying degrees of ab- 
straction. Suggest properties of each of these objects that 
can be measured by each of the four basic types of scales. 


a Store customers. 

b Voter attitudes. 

c Hardness of steel alloys. 

d Preference for a particular common stock. 

e Profitability of various divisions in a company. 


You have been asked by the head of marketing to design 
an instrument by which your private, for-profit school 

can evaluate the quality and value of its various curricula 
and courses. How might you try to ensure that your 
instrument has: 

a Stability? 

b Equivalence? 


nternal consistency? 


c 
d Content validity? 
e Predictive validity? 
f 


Construct validity? 


As part of its bankruptcy restructuring, General Motors 
(GM) launched an ad campaign that revealed glimmers of a 
streamlined GM: fewer brands (Cadillac, Buick, Chevrolet, 
GMC) and fewer models within each brand. 


a What research would you have done to determine which 
vehicle models GM should retain and which it should drop? 


b What would you have measured and with what type of 
measurement scale? 


9 You have been asked to develop an index of student mo- 
rale in your department. 


a What constructs or concepts might you employ? 


b Choose several of the major concepts, and specify their 
dimensions. 


c Select observable indicators that you might use to mea- 
sure these dimensions. 


d How would you compile these various dimensions into a 
single index? 

e How would you judge the reliability and/or validity of 
these measurements? 


Bringing Research to Life 

10 Given that Glacier Symphony has previously measured its 
customer satisfaction by survey, how might Jason assess 
the internal validity of the Glacier questionnaire? 


From Concept to Practice 

11. Using Exhibit 11-3 and one of the case questionnaires 
from the Online Learning Center, match each question to 
its appropriate scale type. For each scale type not repre- 
sented, develop a measurement question that would be of 
that scale type. 


From the Headlines 

12 After more than 75 years in the business, Walsworth Pub- 
lishing’s leadership was well aware of the efforts that go into 
juggling school work and the production of a high-quality 
school yearbook. They examined their communication 
and production processes and decided to implement the 
Information Builders’ WebFOCUS-based reporting portal, 
an online dashboard. Each of the more than 4,000 schools 
had its own dashboard, and the volunteer student staff and 
faculty advisor could access the dashboard in real-time. 
The online dashboard contained notices, progress toward 
deadlines, alerts about missing or corrupt content, and 
more. What should be the research measure(s) to determine 
the effectiveness of the new online dashboard process for 
improving efficiency and quality of each school’s yearbook? 
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neases™ I _______cccctg 
Campbell-Ewald: R-E-S-P-E-C-T ee Ohio Lottery: Innovative Research Design Drives 
Spells Loyalty Winning 


ee Data Development Pebble Beach Co. 
ee Donatos: Finding the New Pizza Ramada Demonstrates Its Personal Best™ 
NCRCC: Teeing Up and New ee USTA: Come Out Swinging 


Strategic Direction 


Yahoo!: Consumer Direct Marries Purchase 
NetConversions Influences Kelley Metrics to Banner Ads 
Blue Book 


*You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 


chapter 12 


learningobjectives 
After reading this chapter, you should understand .. . 


The nature of attitudes and their relationship to behavior. 
The critical decisions involved in selecting an appropriate measurement scale. 


The characteristics and use of rating, ranking, sorting, and other preference scales. 


6 GNo man learns to know his inmost nature 
by introspection, for he rates himself 
sometimes too low, and often too high, by 
his own measurement. Man knows himself 


only by comparing himself with other men; it 
is life that touches his genuine worth. 9 9 


Johann Wolfgang von Goethe, German writer, artist, politician 
(1749-1832) 


bringingresearchtolife....? 


They board the sleek corporate jet in Palm Beach 
and are taken aft to meet with the general manager 
of MindWriter, who is seated at a conference table 
that austerely holds one sheaf of papers and a white 
telephone. 

“T’m Jean-Claude Malraison,” the general manager 
says. “Myra, please sit here . . . and you must be Jason 
Henry. On the flight up from Caracas I read your 
proposal for the CompleteCare project. I intend to 
sign your contract if you answer one question to my 
satisfaction about the schedule. 

“I took research methods in college and didn’t like 
it, so you talk fast, straight, and plainly unless we both 
decide we need to get technical. If the phone rings, 
ignore it and keep talking. When you answer my one 
question, I’ll put you off the plane in the first Florida 
city that has a commercial flight back to... to...” 

“This is Palm Beach, Jean-Claude,’ says the 
steward. 

“What I don’t like is that you are going to hold 
everything up so you can develop a scale for the 
questionnaire. Scaling is what I didn’t like in research. 
It is complicated and it takes too much time. Why can’t 
you use some of the scales our marketing people have 
been using? Why do you have to reinvent the wheel?” 
The manager is looking toward Myra. 

“Our research staff agrees with us that it would be 
inappropriate to adapt surveys developed for use in our 
consumer products line,” says Myra smoothly. 

“OK. Computers are not the same as toaster ovens 
and VCRs. Gotcha. Jason, what is going to be different 
about the scales you intend to develop?” 

“When we held focus groups with your customers, 
they continually referred to the need for your product 
service to ‘meet expectations’ or ‘exceed expectations.’ 
The hundredth time we heard this we realized. . .” 


“Tt’s our company credo, “‘Underpromise and exceed 
expectations.” 

“Well, virtually none of the scales developed for 
customer satisfaction deal with expectations. We 
want a scale that ranges in five steps from ‘Met few 
expectations’ to ‘Exceeded expectations,’ but we don’t 
know what to name the in-between intervals so that the 
psychological spacing is equal between increments. 
We think ‘Met many expectations’ and “Met most 
expectations’ and “Fully met expectations’ will be OK, 
but we want to be sure.” 

“You are not being fussy here, are you, Jason?” 

“No. Because of the way you are running your service 
operation, we want great precision and reliability.” 

“Justify that, please, Myra.” 

“Well, Jean-Claude, besides setting up our own repair 
force, we have contracted with an outside organization 
to provide repairs in certain areas, with the intention after 
six months of comparing the performance of the inside 
and outside repair organizations and giving the future 
work to whoever performs better. We feel that such an 
important decision, which involves the job security of 
MindWriter employees, must have full credibility.” 

“T can accept that. Good.” The manager scribbles his 
signature on the contract. “You'll receive this contract 
in three days, after it has wended its way past the paper 
pushers. Meantime, we’ll settle for a handshake. Nice 
job, so far, Myra. You seem to have gotten a quick start 
with MindWriter. Congratulations, Jason. 

“Turn the plane around and put these folks out where 
they got on. They can start working this afternoon... . 
Gosh, is that the beach out there? It looks great. ve 
got to get some sun one of these days.” 

“You do look pale,” says Myra, sympathetically. 

“Fais gaffe, tu m’fais mal!” he mutters under his 
breath. 
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>Exhibit 12-1 The Scaling Process 


Identify relevant concepts & 
constructs to be measured 


Operationally define 
all constructs 


Choose scale/measurement 
level for constructs 


Choose scale/measurement 
level for concepts 


Identify existing 
scales 


Design a Custom Scale Identify existing 


scales 


Choose a 
construction approach 


Choose an appropriate 
scale type 


Create the scale 


Test the scale 


This chapter covers procedures that will help you understand measurement scales so that you might 
select or design measures that are appropriate for your research. We concentrate here on the problems 
of measuring more complex constructs, like attitudes. Conceptually, we start this process by revisiting 
the research process (see Exhibit 12-1) to understand where the act of scaling fits in the process. 
Scales in business research are generally constructed to measure behavior, knowledge, and attitudes. 
Attitude scales are among the most difficult to construct, so we will use attitudes to develop your un- 
derstanding of scaling. 


> The Nature of Attitudes 


Jason is properly concerned about attitude measurement for the MindWriter study. But what is an 
attitude? There are numerous definitions, but one seems to capture the essence: An attitude is a learned, 
stable predisposition to respond to oneself, other persons, objects, or issues in a consistently favorable 
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iThink can guide you through the communication nuances of 
online focus groups and text based research. Whether you need 
recruiting, hosting, or full moderator support, we have 14+ years 
experience with chat room research. We're comfortable in the 
fast paced online environment and can draw out even the most 
teluctant respondent. Let us explore how the online focus group 
environment can work for your next project. 


Think online with iThink 


www. ithink.com info@ithink.com 


3 7 7 5804 Swiss Avenue, Dallas, TX 75214 
i.think INC 214.855.3777 


>picprofile 


Focus groups are especially useful when you want respondents’ candid thoughts, especially now that the technology for online 
focus groups is more advanced. Conducting a focus group is often a first step in developing a measurement scale that will 
reveal attitudes within a larger quantitative project. As iThink notes, a respondent in an online focus group needs the right type 
of moderator to encourage full participation. Its 14 years of experience is a credential that makes it a likely choice. 
www.ithink.com 


or unfavorable way.' Important aspects of this definition include the learned nature of attitudes, their 
relative permanence, and their association with socially significant events and objects. Because an 
attitude is a predisposition, it would seem that the more favorable one’s attitude is toward a product 
or service, the more likely that the product or service will be purchased. But, as we will see, that is not 
always the case. 


Let’s use Myra as an example to illustrate the nature of attitudes: 


1. She is convinced that MindWriter has great talent, terrific products, and superior opportunities 
for growth. 


2. She loves working at MindWriter. 


3. She expects to stay with the firm and work hard to achieve rapid promotions for greater 
visibility and influence. 


The first statement is an example of a cognitively based attitude. It represents Myra’s memories, 
evaluations, and beliefs about the properties of the object. A belief is an estimate (probability) about 
the truth of something. In this case, it is the likelihood that the characteristics she attributes to her work 
environment are true. The statement “I think the cellular market will expand rapidly to incorporate 
radio and video” is also derived from cognition and belief. The second statement above is an affectively 
based attitude. It represents Myra’s feelings, intuition, values, and emotions toward the object. “I love 
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the Yankees” and “I hate corn flakes” are other examples of emotionally oriented attitudes. Finally, 
researchers recognize a third component, conative or behaviorally based attitudes. The concluding 
statement reflects Myra’s expectations and behavioral intentions toward her firm and the instrumental 
behaviors necessary to achieve her future goals. 


The Relationship between Attitudes 
and Behavior 


The attitude—behavior relationship is not straightforward, although there may be close linkages. 
Attitudes and behavioral intentions do not always lead to actual behaviors; and although attitudes 
and behaviors are expected to be consistent with each other, that is not always the case. Moreover, 
behaviors can influence attitudes. For example, marketers know that a positive experience with a 
product or service reinforces a positive attitude or makes a customer question a negative attitude. 
This is one reason that restaurants where you have a bad dining experience may give you a coupon 
for a free meal on your next visit. They know a bad experience contributes mightily to formation of 
negative attitudes. 

Business researchers treat attitudes as hypothetical constructs because of their complexity and the 
fact that they are inferred from the measurement data, not actually observed. These qualifications 
cause researchers to be cautious about the ways certain aspects of measured attitudes predict behavior. 
Several factors have an effect on the applicability of attitudinal research: 


° Specific attitudes are better predictors of behavior than general ones. 


° Strong attitudes (strength is affected by accessibility or how well the object is remembered and 
brought to consciousness, how extreme the attitude is, or the degree of confidence in it) are 
better predictors of behavior than weak attitudes composed of little intensity or topical interest. 


e Direct experiences with the attitude object (when the attitude is formed, during repeated expo- 
sure, or through reminders) produce behavior more reliably. 


e Cognitive-based attitudes influence behaviors better than affective-based attitudes. 
e Affective-based attitudes are often better predictors of consumption behaviors. 


e Using multiple measurements of attitude or several behavioral assessments across time and 
environments improves prediction. 


e The influence of reference groups (interpersonal support, urges of compliance, peer pressure) 
and the individual’s inclination to conform to these influences improves the attitude-behavior 
linkage.” 


Researchers measure and analyze attitudes because attitudes offer insights about behavior. Many 
of the attitude measurement scales used have been tested for reliability and validity, but often we craft 
unique scales that don’t share those standards. An example is an instrument that measures attitudes 
about a particular tourist attraction, product, or candidate, as well as the person’s intention to visit, 
buy, or vote. Neither the attitude nor the behavioral intent instrument, alone or together, is effective in 
predicting the person’s actual behavior if it has not been designed carefully. Nevertheless, managers 
know that the measurement of attitudes is important because attitudes reflect past experience and shape 
future behavior. 


Attitude Scaling 


Attitude scaling is the process of assessing an attitudinal disposition using a number that represents a 
person’s score on an attitudinal continuum ranging from an extremely favorable disposition to an ex- 
tremely unfavorable one. Scaling is the “procedure for the assignment of numbers (or other symbols) 
to a property of objects in order to impart some of the characteristics of numbers to the properties in 
question.’’* Procedurally, we assign numbers to indicants of the properties of objects. Thus, one assigns 
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a number scale to the various levels of heat and cold and calls it a thermometer. To measure the tem- 
perature of the air, you know that a property of temperature is that its variation leads to an expansion or 
contraction of mercury. A glass tube with mercury provides an indicant of temperature change by the 
rise or fall of the mercury in the tube. Similarly, your attitude toward your university could be measured 
on numerous scales that capture indicators of the different dimensions of your awareness, feelings, or 
behavioral intentions toward the school. 


> Selecting a Measurement Scale 


Selecting and constructing a measurement scale requires the consideration of several factors that influ- 
ence the reliability, validity, and practicality of the scale: 


¢ Research objectives. 

e Response types. 

¢ Data properties. 

¢ Number of dimensions. 

¢ Balanced or unbalanced. 

¢ Forced or unforced choices. 
¢ Number of scale points. 


e Rater errors. 


Research Objectives 


Researchers’ objectives are too numerous to list (including, but not limited to, studies of attitude, 
attitude change, persuasion, awareness, purchase intention, cognition and action, actual and repeat 
purchase). Researchers, however, face two general types of scaling objectives: 


¢ To measure characteristics of the participants who participate in the study. 


¢ To use participants as judges of the objects or indicants presented to them. 


Assume you are conducting a study of customers concerning their attitudes toward a change in corpo- 
rate identity (a company logo and peripherals). With the first study objective, your scale would measure 
the customers’ orientation as favorable or unfavorable. You might combine each person’s answers to 
form an indicator of overall orientation. The emphasis in this first study is on measuring attitudinal 
differences among people. With the second objective, you might use the same data, but you are now 
interested in how satisfied people are with different design options. Each participant is asked to choose 
the object he or she favors or the preferred solution. Participants judge which object has more of some 
characteristic or which design solution is closest to the company’s stated objectives. 


Response Types 


Measurement scales fall into one of four general types: rating, ranking, categorization, and sorting. A 
rating scale is used when participants score an object or indicant without making a direct comparison to 
another object or attitude. For example, they may be asked to evaluate the styling of a new automobile on a 
7-point rating scale. Ranking scales constrain the study participant to making comparisons and determin- 
ing order among two or more properties (or their indicants) or objects. Participants may be asked to choose 
which one of a pair of cars has more attractive styling. A choice scale requires that participants choose 
one alternative over another. They could also be asked to rank-order the importance of comfort, ergonom- 
ics, performance, and price for the target vehicle. Categorization asks participants to put themselves or 
property indicants in groups or categories. Asking auto show attendees to identify their gender or ethnic 
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background or to indicate whether a particular prototype design would appeal to a youthful or mature driver 
would require a category response strategy. Sorting requires that participants sort cards (representing con- 
cepts or constructs) into piles using criteria established by the researcher. The cards might contain photos 
or images or verbal statements of product features such as various descriptors of the car’s performance. 


Data Properties 


Decisions about the choice of measurement scales are often made with regard to the data properties 
generated by each scale. In Chapter 11, we said that we classify scales in increasing order of power; 
scales are nominal, ordinal, interval, or ratio. Nominal scales classify data into categories without indi- 
cating order, distance, or unique origin. Ordinal data show relationships of more than and less than but 
have no distance or unique origin. Interval scales have both order and distance but no unique origin. 
Ratio scales possess all four properties’ features. The assumptions underlying each level of scale deter- 
mine how a particular measurement scale’s data will be analyzed statistically. 


Number of Dimensions 


Measurement scales are either unidimensional or multidimensional. With a unidimensional scale, one 
seeks to measure only one attribute of the participant or object. One measure of an actor’s star power is 
his or her ability to “carry” a movie. It is a single dimension. Several items may be used to measure this 
dimension and by combining them into a single measure, an agent may place clients along a linear con- 
tinuum of star power. A multidimensional scale recognizes that an object might be better described 
with several dimensions than on a unidimensional continuum. The actor’s star power variable might be 
better expressed by three distinct dimensions—ticket sales for last three movies, speed of attracting fi- 
nancial resources, and media buzz (media coverage, Tweets, YouTube videos) on the last three movies. 


Balanced or Unbalanced 


A balanced rating scale has an equal number of categories above and below the midpoint. Generally, 
rating scales should be balanced, with an equal number of favorable and unfavorable response choices. 
However, scales may be balanced with or without an indifference or midpoint option. A balanced scale 
might take the form of “very good—good—average—poor—very poor.” An unbalanced rating scale 
has an unequal number of favorable and unfavorable response choices. An example of an unbalanced 
scale that has only one unfavorable descriptive term and four favorable terms is “poor—fair—good— 
very good—excellent.” The scale designer expects that the mean ratings will be near “good” and that 
there will be a symmetrical distribution of answers around that point, but the scale does not allow par- 
ticipants who are unfavorable to express the intensity of their attitude. 

The use of an unbalanced rating scale can be justified in studies in which researchers know in advance 
that nearly all participants’ scores will lean in one direction or the other. Raters are inclined to score at- 
titude objects higher if the objects are very familiar and if they are ego-involved.* Brand-loyal customers 
are also expected to respond favorably. When researchers know that one side of the scale is not likely to 
be used, they try to achieve precision on the side that will most often receive the participant’s attention. 
Unbalanced scales are also considered when participants are known to be either “easy raters” or “hard 
raters.” An unbalanced scale can help compensate for the error of Jeniency created by such raters. 


Forced or Unforced Choices 


An unforced-choice rating scale provides participants with an opportunity to express no opinion 
when they are unable to make a choice among the alternatives offered. A forced-choice rating scale 
requires that participants select one of the offered alternatives. Researchers often exclude the response 
choice “no opinion,” “undecided,” “don’t know,” “uncertain,” or “neutral” when they know that most 
participants have an attitude on the topic. It is reasonable in this circumstance to constrain participants 
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Restaurant Evaluation 


1. In the last week, if you ate away from your residence or work, which of the following types of establishments did 
you patronize? (Check all that apply.) 


Lunch Dinner 
Buffets (Student Dining Room, Golden Corral, Chinese Buffet, etc.) o Oo 
Chain Restaurants (Fridays, Applebee's, Cracker Barrel, etc.) Bo Oo 
Fast Food Restaurants (McDonalds, Burger King, Taco Bell, etc.) Oo Oo 
Local Restaurants (Seasons, Rudy's, Mi Jalapenos, Post95, etc.) Oo Oo 
Pubs/Bars (Buffalo Wild Wings, O’Conner’s, McMurray’s, Founders, etc.) ob Oo 


2. When choosing a restaurant for food, rate the importance of each the following factors on your decision. 
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1 (Low 2 3 4 5 (High- 
Importance) Importance) 
Comfortable seating O O O O O 
Type of seating (booth vs. table vs. bar-round, etc.) O Restaurant Evaluation 
Cleanliness O 
Menu variety O = 
Quality of food O The purpose of this survey is to learn more about student attitudes and perceptions of restaurant and entertainment venues and, 
in particular, of McMurray’s and Paddy’s Back Door Pizza. The survey will take less than 7 minutes. Your participation will help us 

Large portions O better understand how these businesses can serve your future needs. 
Small portions O We appreciate your honest comments, and all information will be kept confidential. 
Competitive price O At the end of the survey you may enter your email in a raffle to win a prize. 
Time waiting to be seated O 
Supporting a local establishment O 
The use of fresh ingredients vs canned or frozen (e) O O Oo OQ 
Food taste Oo Oo Oo oe) O 
Vegan or Vegitarian options O O O O O 


>picorofile 


Online surveys are increasingly common due in large part to their speed in data collection. They also offer versatility for use with various 
types of measurement scales; flexibility in containing not only verbal but graphical, photographic, video, and digital elements; access to 
difficult-to-contact or inaccessible participants; and lower cost of large-sample completion. The visual appearance of the measurement 
scale is very important in getting the participant to click through to completion. This invitation from the researcher was and the opening 
screen of the questionnaire and was designed to encourage participation; notice both the mention of time, and the inducement of an 
incentive. The first screen of the questionnaire indicates two strategies: a multiple-choice, multiple-response, and a multi-item rating 
grid. If you look closely, you can also see a scroll bar on the first screen. Some designers will put only one question to a screen in an 
online questionnaire believing that participants who have to scroll may not fully complete the survey. This survey was designed and 


fielded through SurveyMonkey. www.surveymonkey.com 


so that they focus on alternatives carefully and do not idly choose the middle position. However, when 
many participants are clearly undecided and the scale does not allow them to express their uncertainty, 
the forced-choice scale biases results. Researchers discover such bias when a larger percentage of 
participants express an attitude than did so in previous studies on the same issue. Some of this bias is 
attributable to participants providing meaningless responses or reacting to questions about which they 
have no attitudes (see Chapter 13). This affects the statistical measures of the mean and median, which 
shift toward the scale’s midpoint, making it difficult to discern attitudinal differences throughout the 
instrument.* Understanding neutral answers is a challenge for researchers. In a customer satisfaction 
study that focused on the overall satisfaction question with a company in the electronics industry, an 
unforced scale was used. Study results, however, revealed that 75 percent of those in the “neutral” par- 
ticipant group could be converted to brand loyalists if the company excelled (received highly favorable 
ratings) on only 2 of the 26 other scaled questions in the study.® Thus, the participants in the neutral 
group weren’t truly neutral, and a forced-choice scale would have revealed the desired information. 


Number of Scale Points 


What is the ideal number of points for a rating scale? Academics and practitioners often have dogmatic 
reactions to this question, but the answer is more practical: A scale should be appropriate for its purpose. 
For a scale to be useful, it should match the stimulus presented and extract information proportionate to 
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>Exhibit 12-2 Characteristics of Scale Types 


Characteristic Dichotomous Multiple Choice Checklist Rating Ranking Free Response 


Nominal Nominal, ordinal, Nominal Ordinal or Ordinal 
or ratio interval 


Nominal or 
ratio 


Type of Scale 


Desired Number 
of Participant 
Answers 


the complexity of the attitude object, concept, or construct. A product that requires little effort or thought 
to purchase, is habitually bought, or has a benefit that fades quickly (low-involvement products) can be 
measured generally with a simple scale. A 3-point scale (better than average—average—worse than 
average) is probably sufficient for a deodorant, a fast-food burger, gift-wrapping, or a snack. There is 
little support for choosing a scale with 5 or more points in this instance. But when the product is com- 
plex, plays an important role in the consumer’s life, and is costly (e.g., financial services, luxury goods, 
automobiles, and other high-involvement products), a scale with 5 to 11 points should be considered. 

As we noted in Chapter 11, the characteristics of reliability and validity are important factors af- 
fecting measurement decisions. First, as the number of scale points increases, the reliability of the 
measure increases.’ Second, in some studies, scales with 11 points may produce more valid results 
than 3-, 5-, or 7-point scales. Third, some constructs require greater measurement sensitivity and the 
opportunity to extract more variance, which additional scale points provide. Fourth, a larger number of 
scale points are needed to produce accuracy when using single-dimension versus multiple-dimension 
scales.’ Finally, in cross-cultural measurement, the cultural practices may condition participants to a 
standard metric—a 10-point scale in Italy, for example. Exhibit 12-2 previews the scales discussed in 
this chapter along with the characteristics of scale types described in Chapter 11. 


Rater Errors 


The value of rating scales depends on the assumption that a person can and will make good judgments. 
Before accepting participants’ ratings, we should consider their tendencies to make errors of central 
tendency and halo effect.'° Some raters are reluctant to give extreme judgments, and this fact accounts 
for the error of central tendency. Participants may also be “easy raters” or “hard raters,” making what 
is called an error of leniency. These errors most often occur when the rater does not know the object 
or property being rated. To address these tendencies, researchers can: 


e Adjust the strength of descriptive adjectives. 


e Space the intermediate descriptive phrases farther apart. 


e Provide smaller differences in meaning between the steps near the ends of the scale than be- 
tween the steps near the center. 


e Use more points in the scale. 


The halo effect is the systematic bias that the rater introduces by carrying over a generalized impression 
of the subject from one rating to another. An instructor expects the student who does well on the first ques- 
tion of an examination to do well on the second. You conclude a report is good because you like its form, or 
you believe someone is intelligent because you agree with him or her. Halo is especially difficult to avoid 
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when the property being studied is not clearly defined, is not easily observed, is not frequently discussed, 
involves reactions with others, or is a trait of high moral importance.'' Ways of counteracting the halo effect 
include having the participant rate one trait at a time, revealing one trait per page (as in an Internet survey, 
where the participant cannot return to change his or her answer), or periodically reversing the terms that 
anchor the endpoints of the scale, so positive attributes are not always on the same end of each scale. 


> Rating Scales 


In Chapter 11, we said that questioning is a widely used stimulus for measuring concepts and con- 
structs. For example, a researcher asks questions about participants’ attitudes toward the taste of a soft 
drink. The responses are “thirst quenching,” “sour,” “strong bubbly,” “orange taste,” and “syrupy.” 
These answers alone do not provide a means of discerning the degree of favorability and thus would 
be of limited value to the researcher. However, with a properly constructed scale, the researcher could 
develop a taste profile for the target brand. We use rating scales to judge properties of objects without 
reference to other similar objects. These ratings may be in such forms as “like—dislike,” “approve— 
indifferent—disapprove,” or other classifications using even more categories. 

Examples of rating scales we discuss in this section are shown in Exhibit 12-2. Since this exhibit 
amplifies the overview presented in this section, we will refer you to the exhibit frequently.’ 


Simple Attitude Scales 


The simple category scale (also called a dichotomous scale) offers two mutually exclusive response 
choices. In Exhibit 12-3 they are “yes” and “no,” but they could just as easily be “important” and 
“unimportant,” “agree” and “disagree,” or another set of discrete categories if the question were 


5) 


>Exhibit 12-3 Sample Rating Scales 


Simple Category Scale 
(dichotomous) 
data: nominal 


Multiple-Choice, 
Single-Response Scale 
data: nominal 


“What newspaper do you read most often for financial news?” 


(] East City Gazette 
L] West City Tribune 


[_] Regional newspaper 
L_] National newspaper 
[_] Other (specify: 


Multiple-Choice, 
Multiple-Response 
Scale (checklist) 
data: nominal 


Likert Scale “The Internet is superior to traditional libraries for comprehensive searches.” 

Summated Rating STRONGLY NEITHER AGREE STRONGLY 

data: interval AGREE AGREE NOR DISAGREE DISAGREE DISAGREE 
(5) (4) (3) (2) (1) 
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>Exhibit 12-3 Cont'd 


Semantic Differential 
Scale 
data: interval 


Numerical Scale 
data: ordinal or* 
interval 


Multiple Rating 
List Scale 

data: ordinal or 
interval 


Constant-Sum Scale 
data: ratio 


Stapel Scale 
data: ordinal or* 
interval 


Graphic Rating Scale 
data: ordinal or* 
interval or ratio 


*In Chapter 11 we noted that researchers differ in the ways they treat data from certain scales. If you are unable to establish the linearity of 
the measured variables or you cannot be confident that you have equal intervals, it is proper to treat data from these scales as ordinal. 
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EXTREMELY 
FAVORABLE 


Employee’s cooperation in teams ___ 
Employee’s knowledge of task ___ 
Employee’s planning effectiveness ___ 


EXTREMELY 


S q » UNFAVORABLE 


“Taking all the supplier characteristics we’ve just discussed and now considering 
cost, what is their relative importance to you (dividing 100 units between)”: 


Being one of the lowest-cost suppliers 


All other aspects of supplier performance 


Sum} 100 


“How likely are you to recommend CompleteCare to others?” (Place an X at the 
position along the line that best reflects your judgment.) 


VERY LIKELY | VERY UNLIKELY 


| 
| 
| 
| 


(alternative with graphic) 
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>picprofile 


Can you use social media research to discover a new pet food? Del Monte, the maker of Snausages, certainly thinks so. 
Using social media as a platform for launching discussions, Del Monte invited more than 300 dog lovers to its site and asked 
them a series of questions, including “What does your dog eat for breakfast?” “Would you buy more treats for your dog if 
they contained vitamins and minerals?” A large number of the participants indicated their dogs preferred eggs and bacon for 
breakfast. These dog owners also tried to use foods that were rich in omega3, antioxidants, and vitamins. Using this research, 
Del Monte created Snausages Breakfast Bites, shaped like fried eggs and bacon strips, and fortified to be nutritious with extra 
calcium, antioxidants, and Omega 3 and 6. It took just six weeks to bring this new snack food to market. Today you can find 


Snausages Breakfast Bites in every pet store in the country. www.snausages.com 


different. This response strategy is particularly useful for demographic questions or where a dichoto- 
mous response is adequate. 

When there are multiple options for the rater but only one answer is sought, the multiple-choice, 
single-response scale is appropriate. Our example has five options. The primary alternatives should 
encompass 90 percent of the range, with the “other” category completing the participant’s list. When 
there is no possibility for an “other” response or exhaustiveness of categories is not critical, the “other” 
response may be omitted. Both the multiple-choice, single-response scale and the simple category scale 
produce nominal data. 

A variation, the multiple-choice, multiple-response scale (also called a checklist), allows the rater 
to select one or several alternatives. In the example in Exhibit 12-3, we are measuring seven items with 
one question, and it is possible that all seven sources for home design were consulted. The cumulative 
feature of this scale can be beneficial when a complete picture of the participant’s choice is desired, 
but it may also present a problem for reporting when research sponsors expect the responses to sum to 
100 percent. This scale generates nominal data. 

Simple attitude scales are easy to develop, are inexpensive, and can be designed to be highly spe- 
cific. They provide useful information and are adequate if developed skillfully. There are also weak- 
nesses. The design approach is subjective. The researcher’s insight and ability offer the only assurance 
that the items chosen are a representative sample of the universe of attitudes about the attitude object. 
We have no evidence that each person will view all items with the same frame of reference as will other 
people. Although such scales are frequently used, there has been a great effort to develop construction 
techniques that overcome some of their deficiencies. 


278 


>part Ill. The Sources and Collection of Data 


Likert Scales 


The Likert scale, developed by Rensis Likert (pronounced Lick-ert), is the most frequently used varia- 
tion of the summated rating scale. Summated rating scales consist of statements that express either a 
favorable or an unfavorable attitude toward the object of interest. The participant is asked to agree or 
disagree with each statement. Each response is given a numerical score to reflect its degree of attitudinal 
favorableness, and the scores may be summed to measure the participant’s overall attitude. Summation 
is not necessary and in some instances may actually be misleading, as our caution below clearly shows. 

In Exhibit 12-3, the participant chooses one of five levels of agreement. This is the traditional Likert 
scale because it meets Likert’s rules for construction and testing. The numbers indicate the value to be 
assigned to each possible answer, with | the least favorable impression of Internet superiority and 5 
the most favorable. Likert scales also use 7 and 9 scale points. Technically, this is known as a Likert- 
type scale since its construction is often less rigorous. However, the advantages of the 7- and 9- point 
scales are a better approximation of a normal response curve and extraction of more variability among 
respondents. The values for each choice are normally not printed on the instrument, but they are shown 
in Exhibit 12-4 to illustrate the scoring system. 

The Likert scale has many advantages that account for its popularity. It is easy and quick to con- 
struct.'* Conscientious researchers are careful that each item meets an empirical test for discriminating 
ability between favorable and unfavorable attitudes. Likert scales are probably more reliable and pro- 
vide a greater volume of data than many other scales. The scale produces interval data. 

Originally, creating a Likert scale involved a procedure known as item analysis. In the first step, a 
large number of statements were collected that met two criteria: (1) Each statement was relevant to the 
attitude being studied; (2) each was believed to reflect a favorable or unfavorable position on that attitude. 
People similar to those who are going to be studied were asked to read each statement and to state the 
level of their agreement with it, using a 5-point scale. A scale value of | indicated a strongly unfavor- 
able attitude (strongly disagree). The other intensities were 2 (disagree), 3 (neither agree nor disagree), 
4 (agree), and 5 (strongly agree), a strongly favorable attitude (see Exhibit 12-3). To ensure consistent 
results, the assigned numerical values are reversed if the statement is worded negatively (1 is always 
strongly unfavorable and 5 is always strongly favorable). Each person’s responses are then added to 
secure a total score. The next step is to array these total scores and select some portion representing the 
highest and lowest total scores (generally defined as the top and bottom 10 to 25 percent of the distribu- 
tion). The middle group (50 to 80 percent of participants) are excluded from the subsequent analysis. 

The two extreme groups represent people with the most favorable and least favorable attitudes to- 
ward the attitude being studied. These extremes are the two criterion groups by which individual items 
are evaluated. Item analysis assesses each item based on how well it discriminates between those 
persons whose total score is high and those whose total score is low. It involves calculating the mean 
scores for each scale item among the low scorers and high scorers. The mean scores for the high-score 
and low-score groups are then tested for statistical significance by computing f values. (In evaluating 
response patterns of the high and low groups to the statement “My digital camera’s features are excit- 
ing,” we secure the results shown in Exhibit 12-4.) After finding the ¢ values for each statement, they 
are rank-ordered, and those statements with the highest t values are selected. The 20 to 25 items that 
have the highest ¢ values (statistically significant differences between mean scores) are selected for in- 
clusion in the final scale.'* Researchers have found that a larger number of items for each attitude object 
improve the reliability of the scale. As an approximate indicator of a statement’s discrimination power, 
one authority also suggests using only those statements whose f¢ value is 1.75 or greater, provided there 
are 25 or more subjects in each group.’° 

Although item analysis is helpful in weeding out attitudinal statements that do not discriminate well, 
the summation procedure causes problems for researchers. The following example on website banner 
ads shows that the same summated score can mean different things: 


1. This banner ad provides the relevant information I expect. 
2. I would bookmark this site to use in the future. 
3. This banner ad is annoying. 


4. I would click for deeper links to discover more details. 
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>Exhibit 12-4 Evaluating a Scale Statement by Item Analysis 

For the statement “My digital camera’s features are exciting,” we select the data from the bottom 25 percent of the distribution (low total 
score group) and the top 25 percent (high total score group). There are 73 people in each group. The remaining 50 percent of the middle of 
the distribution is not considered for this analysis. 


Low Total Score Group High Total Score Group 


Response Categories f fX f 


@® {Strongly agree 5 15 75 5 22 110 550 
Agree 4 4 16 64 4 30 120 480 
Undecided 3 29 87 261 3 15 45 135 
Disagree 2 22 44 838 2 8 16 
Strongly disagree} 1 15 15 15 1 _2 _2 sae? 
Total 73 iar 8 ———@)-——_ 7 285 1,183 

ny =X, =X(FX). ny Xy =X(EX)y 
x= ae 2.42 ~«€ @ > X, = 3.90 
=X, — X,)? = 503 IG ae ® P 3X, — X,)? = 1,183 ieee 
73 73 
= 73.84 = 70.33 
t Xu = XL 
(22 sa ae © 
n(n — 1) 
- 3.90 — 2.42 
: 70.33 + 73.84 
TES = 1) 
— (S192 © 


Legend: 


@ For each of the response categories, the scale’s value (X) is multiplied by the frequency or number of participants (f) who chose 
that value. These values produce the product (fX). This number is then multiplied by X. For example, there are 3 participants in the 
low-score group who scored a 5 (strongly agreed with the statement): (FX) = (3) (6) = 15; (X )(fX) = (6) (15) = 75. 


® The frequencies and products are summed. 
@ Amean score for each group is computed. 
@ Deviation scores are computed, squared, and summed as required by the formula. 


© The data are tested in a modified t-test that compares high- and low-scoring groups for the item. Notice the mean scores in the 
numerator of the formula. 


© The calculated value is compared with a criterion, 1.75. If the calculated value (in this case, 8.92) is equal to or exceeds the 
criterion, the statement is said to be a good discriminator of the measured attitude. (If it is less than the criterion, we would 
consider it a poor discriminator of the target attitude and delete it from the measuring instrument.) We then select the next 
measurement item and repeat the process. 


If a 5-point scale is used, the maximum favorable score would be 20 (assuming 5 is assigned to the 
strongly agree response and question 3, a negation, is reverse-scored). Approximately one-half of 
the statements are worded favorably and the other half unfavorably to safeguard against halo effects. 
The problem of summation arises because different patterns are concealed by the same total score. One 
participant could find the website’s ad relevant, worth returning to, and somewhat pleasing but not 
desire deeper information, whereas another could find the ad annoying but have favorable attitudes on 
the other three questions, thereby producing the same total score. 
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Maritz Discovers Better Way to Measure Customer Satisfaction 


Customer satisfaction in the service industries has long been 
measured by asking for a rating on overall satisfaction, fol- 
lowed by ratings of satisfaction with various influential attributes. 
These ratings are usually provided on a 5- or 7-point scale. Re- 
searchers at Maritz Research, long a leader in ground-breaking 
research methodology, believe they have discovered a better 
approach. They’ve named the approach “Make or Break” cus- 
tomer satisfaction. 

The new questionnaire moves to a non-compensatory 
model in asking questions. Participants, at a hotel for exam- 
ple, are asked first to scale their overall satisfaction. Then, 


1. Please indicate how satisfied you were with your (blank 
experience) using the scale below: [1]Completely Satis- 
fied, [2}/Somewhat Satisfied, [8]Neither Satisfied Nor 
Dissatisfied, [4/Somewhat Dissatisfied, [5]Dissatisfied. 
2a. If 1 to 3, ask: Was the performance on any of these 
aspects so good as, all by itself, to make your overall 
blank experience) satisfactory? 

2b. If 4 or 5, ask: Was the performance on any of these 
aspects so bad as, all by itself, to make your overall 
blank experience) unsatisfactory? 


rather than asking satisfied customers to evaluate a series List of aspects followed. 

of attributes individually using that same scale, they are 

asked to identify from a series of attributes if ONE made their Only after this initial gridded checklist question are respon- 
experience so wonderful as to make them ignore any defi- dents asked to evaluate any attributes not mentioned in 2a or 2b 
ciencies in other attributes. Dissatisfied customers are treated on the provided scale. 

similarly. They are presented the same attributes but asked to Maritz has conducted several split-sample tests, in several 
identity if ONE made their experience so horrendous that no industries, that demonstrate the new survey approach is able to 
positives in the other attributes could save the overall experi- better predict customer return behavior. It also provides manag- 
ence from being terrible. Only after this initial checklist ques- ers with evidence of the importance, either negative or positive, 
tion are they asked to evaluate remaining attributes on the of critical customer service attributes. 

provided scale. www.maritzresearch.com 


Semantic Differential Scales 


The semantic differential (SD) scale measures the psychological meanings of an attitude object using 
bipolar adjectives. Researchers use this scale for studies such as brand and institutional image. The 
method consists of a set of bipolar rating scales, usually with 7 points, by which one or more partici- 
pants rate one or more concepts on each scale item. The SD scale is based on the proposition that an 
object can have several dimensions of connotative meaning. The meanings are located in multidimen- 
sional property space, called semantic space. Connotative meanings are suggested or implied mean- 
ings, in addition to the explicit meaning of an object. For example, a roaring fire in a fireplace may 
connote romantic as well as its more explicit meaning of burning flammable material within a brick 
kiln. One restaurant trying to attract patrons on slow Tuesday evenings offered a special Tuesday menu 
and called it “down home cooking.” Yankee pot roast, stew, and chicken pot pie, although not its usual 
cuisine, carried the connotative meaning of comfort foods and brought patrons into the restaurant, mak- 
ing Tuesday one of the busiest nights of the week. Advertisers, salespeople, and product and package 
designers have long known that they must use words, shapes, associations, and images to activate a 
person’s connotative meanings. 

Osgood and his associates developed the semantic differential method to measure the psycho- 
logical meanings of an object to an individual.'° They produced a list of 289 bipolar adjective 
pairs, which were reduced to 76 pairs and formed into rating scales for attitude research. Their 
analysis allowed them to conclude that semantic space is multidimensional rather than unidimen- 
sional. Three factors contributed most to meaningful judgments by participants: (1) evaluation, 
(2) potency, and (3) activity. These concepts from the historical thesaurus study (Exhibit 12-5) 
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>Exhibit 12-5 Results of the Thesaurus Study 


Evaluation (E) Potency (P) Activity (A) 
Good-bad Hard-soft Active-passive 
Positive-negative Strong-weak Fast-slow 
Optimistic-pessimistic Heavy-light Hot-cold 
Complete-incomplete Masculine-feminine Excitable-calm 
Timely—untimely Severe—lenient 


Tenacious-yielding 


Subcategories of Evaluation 


Meek Goodness Dynamic Goodness Dependable Goodness Hedonistic Goodness 
Clean-dirty Successful-unsuccessful True-false Pleasurable—painful 
Kind-cruel High-low Reputable—disreputable Beautiful-ugly 
Sociable—unsociable Meaningful-meaningless Believing-skeptical Sociable—unsociable 
Light-dark Important-unimportant Wise-foolish Meaningful-meaningless 
Altruistic—egotistical Progressive-regressive Healthy-sick 

Grateful-ungrateful Clean-dirty 


Beautiful-ugly 


Harmonious—dissonant 


Source: Adapted from Charles E. Osgood, G. J. Suci, and P. H. Tannenbaum, The Measurement of Meaning (Urbana: University of Illinois 
Press, 1957), table 5, pp. 52-61. 


illustrate the wide applicability of the technique to persons, abstract concepts, events, institutions, 
and physical objects.'’ 

Researchers have followed a somewhat different approach to SD scales than did the original 
study advocates. They have developed their own adjectives or phrases and have focused on the 
evaluative dimension more often (which might help explain the popularity of the Likert scale). 
The positive benefit is that the scales created have been adapted to specific management ques- 
tions. One study explored a retail store image using 35 pairs of words or phrases classified into 
eight groups. These word pairs were especially created for the study. Excerpts from this scale are 
presented in Exhibit 12-6. Other categories of scale items were “general characteristics of the 
company,” “physical characteristics of the store,” “prices charged by the store,” “store person- 
nel,” “advertising by the store,” and “your friends and the store.” Since the scale pairs are closely 
associated with the characteristics of the store and its use, one could develop image profiles of 
various stores. 

The semantic differential has several advantages. It is an efficient and easy way to secure attitudes 
from a large sample. These attitudes may be measured in both direction and intensity. The total set of 
responses provides a comprehensive picture of the meaning of an object and a measure of the person 
doing the rating. It is a standardized technique that is easily repeated but escapes many problems of 
response distortion found with more direct methods. It produces interval data. Basic instructions for 
constructing an SD scale are found in Exhibit 12-7. 
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>Exhibit 12-6 Adapting SD Scales for Retail Store Image Study 


Convenience of Reaching the Store from Your Location 


Nearby 

Short time required to reach store 
Difficult drive 

Difficult to find parking place 


Convenient to other stores | shop 


Products Offered 


Distant 

Long time required to reach store 
Easy drive 

Easy to find parking place 


Inconvenient to other stores | shop 


Wide selection of different 
kinds of products 


Fully stocked 
Undependable products 
High quality 

Numerous brands 


Unknown brands 


Limited selection of different 
kinds of products 


Understocked 
Dependable products 
Low quality 

Few brands 


Well-known brands 


Source: Robert F. Kelly and Ronald Stephenson, “The Semantic Differential: An Information Source for Designing Retail Patronage 


Appeals,” Journal of Marketing 31 (October 1967), p. 45. 


>Exhibit 12-7 Steps in Constructing an SD Scale 


1. Select the concepts: nouns, noun phrases, or nonverbal stimuli such as visual sketches. Concepts are chosen by judgment and 
reflect the nature of the investigative question. In the MindWriter study, one concept might be “Call Center accessibility.” 


2. Select bipolar word pairs or phrase pairs appropriate to your needs. If the traditional Osgood adjectives are used, several criteria 


guide your selection: 


by factor, to improve their reliability. 


Three bipolar pairs are required when using evaluation, potency, and activity. Scores on these individual items can be averaged, 


The scale should be relevant to the concepts being judged. Choose adjectives that allow connotative perceptions to be 


expressed. Irrelevant concept-scale pairings yield neutral midpoint values that convey little information. 


Scales should be stable across raters and concepts. A pair such as “large—small” may be interpreted as denotative when 


judging a physical object such as “automobile” but may be used connotatively with abstract concepts such as “product 


quality.” 


Scales should be linear between polar opposites and pass through the origin. A pair that fails this test is “rugged—delicate,” 


which is nonlinear on the evaluation dimension. When used separately, both adjectives have favorable meanings.* 


3. Create the scoring system and assign a weight to each point on the scale. The negative signs in the original scoring procedure (—3, 
2, —1,0, +1, +2, +8) were found to produce coding errors, and the O point is arbitrary. Most SD scales have 7 points: 7, 6, 5, 


Al), 2, @lme! tl. 


4. As with Likert scales, about half of the adjective pairs are randomly reversed to minimize the halo effect. 


*Charles E. Osgood, G. J. Suci, and P. H. Tannenbaum, The Measurement of Meaning (Urbana: University of Illinois Press, 1957). 


In Exhibit 12-8 we see a scale being used by a panel of corporate leaders evaluating candidates for 
a high-level position in their industry’s lobbying association. The selection of the concepts is driven by 
the characteristics they believe the candidate must possess to be successful in advancing their agenda. 


There are three candidates. 


Based on the panel’s requirements, we choose 10 scales to score the candidates. The letters along 
the left side, which show the relevant attitude dimension, would be omitted from the actual scale, as 
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>Exhibit 12-8 SD Scale for Analyzing Industry Association Candidates 


() 
(P) 
(A) 
(E) 
(P) 
(A) 
(E) 
(P) 
(A) 
(E) 


>Exhibit 12-9 Graphic Representation of SD Analysis 


Evaluation 
Sociable Unsociable 
Progressive Regressive 
True False 
Successful Unsuccessful 
Potency 
Strong Weak 
Tenacious Yielding 
Heavy Light 
Activity 
Active Passive 
Fast Slow 
Hot Cold 


Jones @———® 
Smith easaw~y 
Williams @-------- e 


would the numerical values shown. Note that the evaluation, potency, and activity scales are mixed. 
To analyze the results, the set of evaluation (E) values is averaged, as are those for the potency (P) and 
activity (A) dimensions. 

The data are plotted in a “snake diagram” in Exhibit 12-9. Here the adjective pairs are reordered so 
that evaluation, potency, and activity descriptors are grouped together, with the ideal factor reflected 
by the left side of the scale. 


Numerical/Multiple Rating List Scales 


Numerical scales have equal intervals that separate their numeric scale points, as shown in Exhibit 12-3. 
The verbal anchors serve as the labels for the extreme points. Numerical scales are often 5-point scales 
but may have 7 or 10 points. The participants write a number from the scale next to each item. If numer- 
ous questions about a product’s performance were included in the example, the scale would provide 
both an absolute measure of importance and a relative measure (ranking) of the various items rated. The 
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scale’s linearity, simplicity, and production of ordinal or interval data make it popular for managers and 
researchers. When evaluating a new product concept, purchase intent is frequently measured with a 5- to 
7-point numerical scale, with the anchors being “definitely would buy” and “definitely would not buy.” 

A multiple rating list scale (Exhibit 12-3) is similar to the numerical scale but differs in two ways: 
(1) It accepts a circled response from the rater, and (2) the layout facilitates visualization of the results. 
The advantage is that a mental map of the participant’s evaluations is evident to both the rater and the 
researcher. This scale produces interval data. 


Stapel Scales 


The Stapel scale is used as an alternative to the semantic differential, especially when it is difficult to 
find bipolar adjectives that match the investigative question. In the example in Exhibit 12-3 there are 
three attributes of corporate image. The scale is composed of the word (or phrase) identifying the image 
dimension and a set of 10 response categories for each of the three attributes. 

Fewer response categories are sometimes used. Participants select a plus number for the character- 
istic that describes the attitude object. The more accurate the description, the larger is the positive num- 
ber. Similarly, the less accurate the description, the larger is the negative number chosen. Ratings range 
from +5 to —5, with participants selecting a number that describes the store very accurately to very 
inaccurately. Like the Likert, SD, and numerical scales, Stapel scales usually produce interval data. 


Constant-Sum Scales 


A scale that helps the researcher discover proportions is the constant-sum scale. With a constant-sum 
scale, the participant allocates points to more than one attribute or property indicant, such that they total 
a constant sum, usually 100 or 10. In the Exhibit 12-3 example, two categories are presented that must 
sum to 100. In the restaurant example, the participant distributes 100 points among four categories: 


You have 100 points to distribute among the following characteristics of the Dallas Steakhouse. 
Indicate the relative importance of each attribute: 


___— Food Quality 
____ Atmosphere 
____ Service 
____ Price 

100 TOTAL 


Up to 10 categories may be used, but both participant precision and patience suffer when too many 
stimuli are proportioned and summed. A participant’s ability to add is also taxed in some situations; 
thus, this is not a response strategy that can be effectively used with children or the uneducated. The 
advantage of the scale is its compatibility with percent (100 percent) and the fact that alternatives that 
are perceived to be equal can be so scored—unlike the case with most ranking scales. The scale is used 
to record attitudes, behavior, and behavioral intent. The constant-sum scale produces interval data. 


Graphic Rating Scales 


The graphic rating scale was originally created to enable researchers to discern fine differences. Theoreti- 
cally, an infinite number of ratings are possible if participants are sophisticated enough to differentiate and 
record them. They are instructed to mark their response at any point along a continuum. Usually, the score 
is a measure of length (millimeters) from either endpoint. The results are treated as interval data. The dif- 
ficulty is in coding and analysis. This scale requires more time than scales with predetermined categories. 


Never Xx Always 


Other graphic rating scales (see Exhibit 12-3) use pictures, icons, or other visuals to communicate 
with the rater and represent a variety of data types. Graphic scales are often used with children, whose 
more limited vocabulary prevents the use of scales anchored with words. 


> Ranking Scales 


In ranking scales, the participant directly compares two or 
more objects and makes choices among them. Frequently, 
the participant is asked to select one as the “best” or the 
“most preferred.” When there are only two choices, this 
approach is satisfactory, but it often results in ties when 
more than two choices are found. For example, assume 
participants are asked to select the most preferred among 
three or more models of a product. In response, 40 percent 
choose model A, 30 percent choose model B, and 30 per- 
cent choose model C. Which is the preferred model? The 
analyst would be taking a risk to suggest that A is most pre- 
ferred. Perhaps that interpretation is correct, but 60 percent 
of the participants chose some model other than A. Perhaps 
all B and C voters would place A last, preferring either B 
or C to A. This ambiguity can be avoided by using some of 
the techniques described in this section. 

Using the paired-comparison scale, the par- 
ticipant can express attitudes unambiguously by 
choosing between two objects. Typical of paired com- 
parisons would be the sports car preference example in 
Exhibit 12-10. The number of judgments required in a 


>Exhibit 12-10 Ranking Scales 


Paired-Comparison 
Scale 
data: ordinal 


Forced Ranking Scale “Rank the radar detection features in your order of preference. Place the 
data: ordinal number 1 next to the most preferred, 2 by the second choice, and so forth.” 


____ User programming 
___ Cordless capability 
___ Small size 

____ Long-range warning 
___ Minimal false alarms 


Comparative Scale 
data: ordinal 
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Assume you are 
asked by Galaxy 
Department Stores to 
study the shopping 
habits and prefer- 
ences of teen girls. 
Galaxy is seeking a 
way to compete with 
specialty stores that 
are far more suc- 
cessful in serving 

this market segment. 
Galaxy is considering 
the construction of an 
intrastore boutique 
catering to these 
teens. What measure- 
ment issues would 
determine your con- 
struction of measure- 
ment scales? 
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>Exhibit 12-11 Response Patterns of 200 Heavy Users’ Paired Comparisons on Five 
Alternative Package Designs 
Paired-comparison data may be treated in several ways. If there is substantial consistency, we will find that if A is 
preferred to B, and B to C, then A will be consistently preferred to C. This condition of transitivity need not always be 
true but should occur most of the time. When it does, take the total number of preferences among the comparisons as 
the score for that stimulus. Assume a manager is considering five distinct packaging designs. She would like to know 
how heavy users would rank these designs. One option would be to ask a sample of the heavy-users segment to pair- 
compare the packaging designs. With a rough comparison of the total preferences for each option, it is apparent that 
B is the most popular. 


Designs 
c 
A _ 164* 138 50 70 
B 36 = 54 14 30 
Cc 62 146 = 32 50 
D 150 186 168 = 118 
E 130 170 150 82 = 
Total 378 666 510 178 268 
Rank order 3 1 2 5 4 


“Interpret this cell as 164 of 200 customers preferred suggested design B (column) to design A (row). 


paired comparison is [(7)(n — 1)/2], where n is the number of stimuli or objects to be judged. When 
four cars are evaluated, the participant evaluates six paired comparisons [(4)(3)/2 = 6]. 

In another example we might compare packaging design proposals considered by a brand manager (see 
Exhibit 12-11). Generally, there are more than two stimuli to judge, resulting in a potentially tedious task for 
participants. If 15 suggestions for design proposals are available, 105 paired comparisons would be made. 

Reducing the number of comparisons per participant without reducing the number of objects can 
lighten this burden. You can present each participant with only a sample of the stimuli. In this way, 
each pair of objects must be compared an equal number of times. Another procedure is to choose a 
few objects that are believed to cover the range of attractiveness at equal intervals. All other stimuli 
are then compared to these few standard objects. If 36 automobiles are to be judged, four may be 
selected as standards and the others divided into four groups of eight each. Within each group, the 
eight are compared to each other. Then the 32 are individually compared to each of the four standard 
automobiles. This reduces the number of comparisons from 630 to 240. 

Paired comparisons run the risk that participants will tire to the point that they give ill-considered 
answers or refuse to continue. Opinions differ about the upper limit, but five or six stimuli are not un- 
reasonable when the participant has other questions to answer. If the data collection consists only of 
paired comparisons, as many as 10 stimuli are reasonable. A paired comparison provides ordinal data. 

The forced ranking scale, shown in Exhibit 12-10, lists attributes that are ranked relative to each 
other. This method is faster than paired comparisons and is usually easier and more motivating to the 
participant. With five items, it takes 10 paired comparisons to complete the task, and the simple forced 
ranking of five is easier. Also, ranking has no transitivity problem where A is preferred to B, and B to 
C, but C is preferred to A—although it also forces a false unidimensionality. 

A drawback to forced ranking is the number of stimuli that can be handled by this method. Five ob- 
jects can be ranked easily, but participants may grow careless in ranking 10 or more items. In addition, 
rank ordering produces ordinal data since the distance between preferences is unknown. 

Often the manager is interested in benchmarking. This calls for a standard by which other programs, 
processes, brands, point-of-sale promotions, or people can be compared. The comparative scale is 
ideal for such comparisons if the participants are familiar with the standard. In the Exhibit 12-10 ex- 
ample, the standard is the participant’s previous hair dryer. The new dryer is being assessed relative 


Paired Comparison Increases Hospitality 
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Should Northwest Airlines, Marriott, or Alaskan Airlines attempt 
to attract the business of Americans with disabilities? If so, what 
would it take to capture the segment? Eric Lipp, executive di- 
rector of the Open Doors Organization (ODO), an advocacy 
organization for those with disabilities, sponsored a study to 
find out. High on his agenda was providing an incentive to the 
travel industry to make accommodations to attract the 22 million 
adults with disabilities who have traveled in the last two years on 
63 million trios—and who may want to travel more. “We now es- 
timate that Americans with disabilities currently spent $13.2 bil- 
lion in travel expenditures and that amount would at least double 
[to $27.2 billion] if travel businesses were more attuned to the 
needs of those with disabilities.” 

ODO hired Harris Interactive, a global market research and 
consulting firm best known for The Harris Poll and for pioneer- 
ing the Internet method to conduct scientifically accurate mar- 
ket research. Harris Interactive conducted a hybrid study via 
both online and phone surveys to determine the magnitude of 
the disability travel segment, its purchasing power, and the ac- 
commodations the segment needed to increase travel. “Those 


And how did the firm evaluate all the possible accommodations— 
from Braille safety cards on airplanes to a designated person to 
handle problems in a hotel? Harris Interactive used its proprietary 
COMPASS™ methodology, which uses paired comparisons as 
a measurement tool. “COMPASS™ saves the participant time 
and energy,” explained Light. “Even with a long list, COMPASS™ 
can be done quickly.” In the ODO study, COMPASS™ was used 
twice: once to measure 17 possible airline accommodations and 
once to measure 23 possible hotel accommodations. By hav- 
ing each participant evaluate only a portion of the large number 
of accommodation pairs rather than the full list (186 for airline 
accommodations and 253 for hotel accommodations), each 
question was answered in under four minutes. By using this pro- 
cess with all members of the sample, Harris Interactive is able to 
rank-order the items and measure the magnitude of difference 
between items. This makes it easier for Delta, Marriott, or Alas- 
kan Airlines to make the right choices about accommodations for 
those with disabilities. 


www.opendoorsnfp.org; www.harrisinteractive.com 


To learn more about this research, read the case “Open Doors: 


with disabilities can’t all be reached with one method,” explained 
Laura Light, project director with Harris Interactive. “The nature of 
their physical limitation might preclude one method or the other.” 


Extending Hospitality to Travelers with Disabilities” available from 
the Online Learning Center (www.mhhe.com/cooper1 2e). 


to it. The provision to compare yet other dryers to the standard is not shown in the example but is 
nonetheless available to the researcher. 

Some researchers treat the data produced by comparative scales as interval data since the scoring 
reflects an interval between the standard and what is being compared. We would treat the rank or po- 
sition of the item as ordinal data unless the linearity of the variables in question could be supported. 


> Sorting 


Q-sorts require sorting of a deck of cards into piles that represent points along a continuum. The 
participant—or judge—groups the cards based on his or her response to the concept written on the 
card. Researchers using Q-sort resolve three special problems: item selection, structured or unstructured 
choices in sorting, and data analysis. The basic Q-sort procedure involves the selection of a set of verbal 
statements, phrases, single words, or photos related to the concept being studied. For statistical stability, 
the number of cards should not be less than 60; and for convenience, not be more than 120. After the 
cards are created, they are shuffled, and the participant is instructed to sort the cards into a set of piles 
(usually 7 to 11), each pile representing a point on the judgment continuum. The left-most pile represents 
the concept statements, which are “most valuable,” “favorable,” “agreeable,” and so forth. The right-most 
pile contains the least favorable cards. The researcher asks the participant to fill the center, or neutral, pile 
with the cards about which the participant is indecisive. In the case of a structured sort, the distribution of 
cards allowed in each pile is predetermined. With an unstructured sort, only the number of piles will be 
determined. Although the distribution of cards in most structured sorts resembles a normal distribution, 
there is some controversy about analyzing the data as ranking (ordinal data) versus interval data. 
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MindWriter Scaling 


() Jason has been working on scaling for Mind- 

Writer's CompleteCare project for a week when 
MindWriter\ the request comes to Myra Wines to report her 
progress to MindWriter’s general manager. He has narrowed 
he choice to the three scales in Exhibit 12-12: a Likert scale, 
a numerical rating scale with two verbal anchors, and a hybrid 
expectation scale. All are 5-point scales that are presumed to 
measure at the interval level. 

He needs a statement that can accompany the scale for pre- 
iminary evaluation. Returning to their list of investigative ques- 
ions, he finds a question that seems to capture the essence of 
he repair process: Are customers’ problems resolved? Trans- 
ated into an assertion for the scale, the statement becomes 
“Resolution of problem that prompted service/repair.” He contin- 
ues to labor over the wording of the verbal anchors. Appropriate 
versions of the investigative question are constructed, and then 
the scales are added. 

After consulting with MindWriter’s research staff, Myra and 
Jason discuss the advantages of the scales. Myra suggests it is 
unlikely that CompleteCare would meet none of the customers’ 
expectations. And, with errors of leniency, “none” should be re- 
placed by the term “few” so that the low end of the scale would 
be more relevant. Jason has read a Marketing News article that 


said Likert scales and scales similar to MindWriter’s numeri- 
cal scale frequently produced a heavy concentration of 4s and 
5s—a common problem in customer satisfaction research. 

Others suggest a 7-point scale to remedy this, but Jason 
thinks the term “exceeded” on the expectation scale could com- 
pensate for scores that cluster on the positive end. 

Now ready for a pilot test, Jason decides to compare the 
expectation scale with MindWriter’s numerical rating scale. 
(The typical Likert scale requires that they create more potential 
items than they wanted to use on the online survey.) Using the 
CompleteCare database, names, addresses, emails and phone 
numbers are selected, and 30 customers are chosen at random 
from those who have had recent service. Jason chooses the de- 
layed equivalent forms method for reliability testing (see Chapter 
11). Jason administers the expectation scale followed by the 
numerical scale to half of the participants and the numerical 
scale followed by the expectation scale to the other half. Each 
half sample experiences a time delay. No “order-of-presentation” 
effects are found. Subsequently, Jason correlates the numerical 
satisfaction scores with the expectation scores and plots the 
results, shown in Exhibit 12-13. 

On the problem resolution question, the participants’ scores 
from the satisfaction scale and those from the expectation scale 


>Exhibit 12-12 Alternative Scales Considered for MindWriter 


Likert Scale 


The problem that prompted service/repair was resolved. 


Strongly 


Disagree Disagree 


Neither Agree 
nor Disagree 


Strongly 
Agree 


To what extent are you satisfied that the problem that prompted service/repair was resolved? 


Very 
Dissatisfied 


Resolution of the problem that prompted service/repair. 


Very 
Satisfied 


Met Few Met Some Met Most Met All Exceeded 
Expectations Expectations Expectations Expectations Expectations 
1 2 3 4 5 
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>closeupcont’d 


>Exhibit 12-13 Plot of MindWriter Scale Evaluation are positively correlated. The correlation index is .90 (1.00 is 
a perfect positive correlation). This reveals the equivalence of 
both scales. On another reliability issue, stability, a check of the 
test-retest reliability over a one-week interval produces a cor- 
relation index of .93 for the expectation scale, but the Mind- 
Writer satisfaction scale has a lower index (r = .75). This implies 
that the expectation scale is more likely to produce stable and 
consistent results. Finally, as shown in the plot, the data for the 
resolution question measured on both scales are linear (they 
cluster around a straight line). It is logical to conclude that the 
expectation scale, which was originally thought to have advan- 
tages, performs well in initial tests and is a good replacement for 
MindWriter’s existing scale. 

The decision is made. They will use the new hybrid expec- 
{ 2 3 4 5 tation scale for the MindWriter research on the CompleteCare 
service program. 


5 


Satisfaction scores 


Expectation scores 


The purpose of sorting is to get a conceptual representation of the sorter’s attitude toward the at- 
titude object and to compare the relationships between people. The relative ranking of concepts allows 
researchers to derive clusters of individuals possessing similar preferences. By researchers varying 
the instructions, the technique can be used to describe products, services, behavioral intentions, and a 
host of other applications. In the example below, participants are asked to complete a structured sort 
of cards containing the names of magazines. The scale values and the number of cards in each pile are 
predetermined, although the distribution in this case represents a normal statistical distribution. 


What magazines do you want Singapore Airlines to carry for its in-flight service? 


Most Least 

Preferred Preferred 

10 9 8 7 6 5 4 3 2 1 0 (scale value) 

3 4 td 10 13 16 13 pb 6) ri 4 3 (number of cards per pile) 


> Cumulative Scales 


Total scores on cumulative scales have the same meaning. Given a person’s total score, it is possible 
to estimate which items were answered positively and negatively. A pioneering scale of this type was 
the scalogram. Scalogram analysis is a procedure for determining whether a set of items forms a uni- 
dimensional scale.'* A scale is unidimensional if the responses fall into a pattern in which endorsement 
of the item reflecting the extreme position results in endorsing all items that are less extreme. 

Assume we are surveying opinions regarding a new style of running shoe. We have developed a 
preference scale of four items: 


1. The Airsole is good-looking. 
2. I will insist on Airsole next time because it is great-looking. 
3. The appearance of Airsole is acceptable to me. 


4. I prefer the Airsole style to other styles. 
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>Exhibit 12-14 Ideal Scalogram Response Pattern* 


Participant Score 


X Xx x X 4 
_ Xx Xx X 3 
- - Xx Xx 2 
Xx 1 
0) 
*X = agree; — = disagree. 


Participants indicate whether they agree or disagree. If these items form a unidimensional scale, the 
response patterns will approach the ideal configuration shown in Exhibit 12-14. Item 2 is the most 
extreme position of the four attitude statements. A participant who agrees with item 2 will agree with 
all four items. The items are ordered in the scalogram left to right from most to least extreme. If each 
agreement renders a score of 1, a score of 4 indicates all statements are agreed upon and represents the 
most favorable attitude. Persons with a score of 3 should disagree with item 2 but agree with all others, 
and so on. According to scalogram theory, this pattern confirms that the universe of content (attitude 
toward the appearance of this running shoe) is scalable. 

The scalogram and similar procedures for discovering underlying structure are useful for assessing 
attitudes and behaviors that are highly structured, such as social distance, organizational hierarchies, 
and evolutionary product stages.'? The scalogram is used much less often today, but retains potential 
for specific applications. 


1 Managers know that the measurement of attitudes is 
an important aspect of strategy and often the best tool 
available because attitudes reflect past experience and 
shape future behavior. Attitudes are learned, stable pre- 


be analyzed statistically. Measurement scales are either 
unidimensional or multidimensional. A balanced rating 
scale has an equal number of categories above and below 
the midpoint, whereas an unbalanced rating scale has an 


dispositions to respond to oneself, other persons, ob- 
jects, or issues in a consistently favorable or unfavorable 
way. Attitudes are generally thought to be composed of 
three components: affective, cognitive, and behavioral 
intentions. 


Selecting and constructing a measurement scale requires 
the consideration of several factors that influence the reli- 
ability, validity, and practicality of the scale. Two broad 
research objectives are to measure characteristics of the 
individuals who participate in studies and to use partici- 
pants as judges of the objects or indicants presented to 
them. Measurement scales fall into one of four general 
response types: rating, ranking, categorization, and sort- 
ing. The properties of data are classified in increasing 
order of power—nominal, ordinal, interval, or ratio—which 
determines how a particular measurement scale’s data will 


unequal number of favorable and unfavorable response 
choices. An unforced-choice rating scale provides par- 
ticipants with an opportunity to express no opinion when 
they are unable to make a choice among the alternatives 
offered. A forced-choice scale requires that they select 
one of the offered alternatives. The ideal number of points 
for a rating scale should match the stimulus presented 
and extract information proportionate to the complexity 
of the attitude object. The value of rating scales depends 
on the assumption that a rater can and will make good 
judgments. Errors of central tendency, halo effect, and 
leniency adversely affect a precise understanding of the 
measurement. 


Rating scales have several uses, design features, and 
requirements. The simple category scale offers two mu- 
tually exclusive response choices. The multiple-choice, 


single-response scale offers the rater several options, 
including “other.” The multiple-choice, multiple-response 
scale (also called a checklist) allows the rater to select 
one or several alternatives, thereby providing a cumulative 
feature. 
The Likert scale consists of a series of statements, and 
the participant is asked to agree or disagree with each 
statement. Summation is possible with this scale although 
not necessary and in some instances undesirable. 

The semantic differential (SD) scale measures the psy- 
chological meanings of an attitude object. Researchers 
use this scale for studies of brand and institutional image. 


The method consists of a set of bipolar rating scales, usu- 


ally with 7 points, by which one or more participants rate 
one or more concepts on each scale item. The Stapel 

scale is used as an alternative to the semantic differential, 
especially when it is difficult to find bipolar adjectives that 


match the investigative question. Participants select a plus 


number for the characteristic that describes the attitude 


object. Ratings range from +5 to —5, and participants se- 
lect a number that describes the object very accurately to 
very inaccurately. 

Numerical scales have equal intervals that separate their 
numeric scale points. Verbal anchors serve as the labels 
for the extreme points. Numerical scales are often 5-point 
scales but may have 7 or 10 points. A multiple rating list 
scale is similar to the numerical scale but accepts a circled 
response from the rater, and the layout allows visualization 
of the results. 

A scale that helps the researcher discover proportions 
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100 points among up to 10 categories. The graphic rating 
scale was originally created to enable researchers to discern 
fine differences. Raters check their response at any point 
along a continuum. Other graphic rating scales use pictures, 
icons, or other visuals to communicate with children or 
others whose limited vocabulary prevents the use of scales 
anchored with words. 

Ranking scales allow the participant to compare two or 
more objects and make choices among them. Frequently, 
the participant is asked to select one as the “best” or the 
“most preferred.” When there are only two choices, as with 
the paired-comparison scale, the participant can express 
attitudes unambiguously by choosing between two objects. 
The forced ranking scale lists attributes that are ranked 
relative to each other. This method is faster than paired 
comparisons and more user-friendly. Often the researcher 
is interested in benchmarking. This calls for a standard by 
which training programs, processes, brands, point-of-sale 
purchases, or people can be compared. The comparative 
scale is ideal for such comparisons if the participants are 
familiar with the standard. 

Q-sorts are a form of scaling that requires sorting of a 
deck of cards into piles that represent points along a con- 
tinuum. The purpose of sorting is to get a conceptual repre- 
sentation of the sorter’s attitude toward the attitude object 
and to compare the relationships between people. Given a 
person’s total score, it is possible to estimate which items 
were answered positively and negatively on cumulative 
scales. A pioneering cumulative scale was the scalogram, 

a procedure for determining whether a set of items forms a 


is the constant-sum scale. The participant distributes 


unidimensional scale. 
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>di ion tion 


Terms in Review 
1 Discuss the relative merits of and problems with: 


a Rating and ranking scales. 
b Likert and differential scales. 
c Unidimensional and multidimensional scales. 
Making Research Decisions 

2 Assume you are Menu Foods and you planned a major re- 
search study just prior to the largest pet food recall in our his- 
tory. You plan to proceed with the study and feel you must 
add one or more questions to measure the consumer’s con- 


fidence that your firm will be able to recover. Draft a scale for 
each of the following types that will measure that confidence. 


a Constant-sum scale. 

b Likert-type summated scale. 
c Semantic differential scale. 
d Stapel scale. 

e Forced ranking scale. 


3 An investigative question in your employee satisfaction 
study seeks to assess employee “job involvement.” Create 
a measurement question that uses the following scales: 


a Agraphic rating scale. 
b A multiple rating list. 
Which scale do you recommend and why? 


4 You receive the results of a paired-comparison preference 
test of four soft drinks from a sample of 200 persons. The 
results are as follows: 


Mr. 
Koak Zip Pabze Peepers 
Koak _ 50* 115 35 
Zip 150 = 160 70 
Pabze 85 40 = 45 
Mr. Peepers 165 130 155 _ 


*Read as 50 persons preferred Zip to Koak. 


a How do these brands rank in overall preference in this 
sample? 


b Develop an interval scale for these four brands. 


5 One of the problems in developing rating scales is the choice 
of response terms to use. Below are samples of some widely 
used scaling codes. Do you see any problems with them? 


a Yes—Depends—No 
b Excellent—Good—Fair—Poor 


ce Excellent—Good—Average—Fair—Poor 


d Strongly Approve—Approve—Uncertain—Disapprove— 
Strongly Disapprove 


6 You are working on a consumer perception study of four 
brands of bicycles. You will need to develop measurement 
questions and scales to accomplish the tasks listed below. 
Be sure to explain which data levels (nominal, ordinal, 
interval, ratio) are appropriate and which quantitative tech- 
niques you will use. 


a Prepare an overall assessment of all the brands. 
b Provide a comparison of the brands for each of the 
following dimensions: 
(1) Styling 
(2) Durability 
(3) Gear quality 
(4) Brand image 
7 Below is a Likert-type scale that might be used to evalu- 
ate your opinion of the educational degree program in 
which you are enrolled. There are five response categories: 
Strongly Agree, Agree, Neither Agree nor Disagree, Dis- 
agree, and Strongly Disagree. If Strongly Agree (SA) repre- 


sents the most positive attitude, how would you value the 
items below? Record your answers to the items. 


a This program is not very challenging. SA AN D SD 

b The general level of teaching is good. SA AN D SD 

c | really think | am learning a lot from SA AN D SD 
this program. 

d Students’ suggestions are given SA AN D SD 
little attention here. 

e This program does a good job of SA AN D SD 
preparing one for a career. 

f This program is below my expectations. SA A N D SD 


In what two different ways could such responses be 
used? What would be the purpose of each? 


Bringing Research to Life 
8 What is the basis of Jason and Myra’s argument for the need 
of an arbitrary scale to address customer expectations? 


From Concept to Practice 
9 Using the response strategies within Exhibit 12-1 or 12-10, 
which would be appropriate and add insight to under- 
standing the various indicants of student demand for the 
academic program in which the students are enrolled? 


From the Headlines 

10 The third generation iPad disappointed many who had ex- 
pected not only a higher resolution screen (delivered) but 
Siri, Apple’s much ballyhooed speak-and-it-responds app 
developed for the iPhone 4S (not delivered). Apple indicated 
it wasn’t included because Siri for the iPad wasn’t ready. 
Design the question(s) that could have revealed expecta- 
tions of iPad3 devotees. 
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downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 


chapter 13 


learningobjectives 
After reading this chapter, you should understand . . . 
The link forged between the management dilemma and the communication instrument by the management-research 
question hierarchy. 
The influence of the communication method on instrument design. 
The three general classes of information and what each contributes to the instrument. 


The influence of question content, question wording, response strategy, and preliminary analysis planning on 
question construction. 


Each of the numerous question design issues influencing instrument quality, reliability, and validity. 
Sources for measurement questions. 


The importance of pretesting questions and instruments. 


6 GWAP (mobile browser-based) surveys offer full survey 
functionality (including multimedia) and can be accessed 
from any phone with a web browser (which is roughly 90 
percent of all mobile devices). As an industry we need 
to get comfortable with mobile survey formats because 


there are fundamental differences in survey design and 
we also need to be focused on building our mobile 
capabilities as part of our sampling practice. 9 9 


Kristin Luck, president, 
Decipher 


bringingresearcnhtolife 


“How is the Albany questionnaire coming?” asks Jason 
as he enters Sara’s office. 

“The client approved the investigative questions this 
morning. So we are ready to choose the measurement 
questions and then write the questionnaire,” shares 
Sara, glancing up from her computer screen. “I was just 
checking our bank of pretested questions. I’m looking 
for questions related to customer satisfaction in the 
medical field.” 

“If you are already searching for appropriate 
questions, you must have the analysis plan drafted. Let 
me see the dummy tables you developed,” requests 
Jason. “I'll look them over while you’re scanning.” 

Sara hands over a sheaf of pages. Each has one 
or more tables referencing the desired information 
Each table 
diagnostics that would be needed to generate the 
table. 

As the computer finishes processing, Sara scans 


variables. indicates the statistical 


the revealed questions for appropriate matches to 
Albany’s information needs. “At first glance, it looks 
like there are several multiple-choice scales and 
ranking questions we might use. But I’m not seeing a 
rating scale for overall satisfaction. We may need to 
customize a question just for Albany.” 


“Custom designing a question is expensive. Before 
you make that choice,” offers Jason, “run another 
query using CardioQuest as a keyword. A few years 
ago, I did a study for that large cardiology specialty 
in Orlando. I’m sure it included an overall satisfaction 
scale. It might be worth considering.” 

Sara types CardioQuest and satisfaction, and then 
waits for the computer to process her request. “Sure 
enough, he’s right again,” murmurs Sara. “How do you 
remember all the details of prior studies done eons ago?” 
she asks, throwing the purely hypothetical question at 
Jason. But Sara swivels to face Jason, all senses alert 
when she hears his muffled groan. 

Jason frowns as he comments, “You have far more 
analytical diagnostics planned than would be standard 
for a project of this type and size, Sara. For example, 
are Tables 2, 7, and 10 really necessary?” Jason 
pauses but doesn’t allow time for Sara to answer. “To 
stay within budget, we are going to have to whittle 
down the analysis phase of the project to what is 
essential. Let’s see if we can reduce the analysis plan 
to something that we both can live with. Now, walk 
me through what you think you’ll reveal by three-way 
cross-tabulating these two attitudinal variables with 
the education variable.” 
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>Exhibit 13-1 Overall Flowchart for Instrument Design 
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New researchers often want to draft questions immediately. Their enthusiasm makes them reluctant to 
go through the preliminaries that make for successful surveys. Exhibit 13-1 is a suggested flowchart 
for instrument design. The procedures followed in developing an instrument vary from study to study, 
but the flowchart suggests three phases. Each phase is discussed in this chapter, starting with a review 
of the research question hierarchy. 


> Phase 1: Revisiting the Research Question 
Hierarchy 


The management-research question hierarchy is the foundation of the research process and also of suc- 
cessful instrument development (see Exhibit 13-2). By this stage in a research project, the process of 
moving from the general management dilemma to specific measurement questions has traveled through 
the first three question levels: 


1. Management question—the dilemma, stated in question form, that the manager needs resolved. 


2. Research question(s )—the fact-based translation of the question the researcher must answer to 
contribute to the solution of the management question. 


3. Investigative questions—specific questions the researcher must answer to provide sufficient 
detail and coverage of the research question. Within this level, there may be several questions 
as the researcher moves from the general to the specific. 

4. Measurement questions—questions participants must answer if the researcher is to gather the 
needed information and resolve the management question. 


In the Albany Outpatient Laser Clinic study, the eye surgeons would know from experience the types 
of medical complications that could result in poor recovery. But they might be far less knowledgeable 
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>Exhibit 13-2 Flowchart for Instrument Design: Phase 1 


Select Scale Type 


(nominal, ordinal, interval, ratio) 


Select Communication 
Approach 


(personal, phone, electronic, mail) 


Select Process Structure 
(structured, unstructured, 
combination; disguised vs. undisguised) 


about what medical staff actions and attitudes affect client recovery and perception of well-being. 
Coming up with an appropriate set of information needs in this study will take the guided expertise of 
the researcher. Significant exploration would likely have preceded the development of the investiga- 
tive questions. In the project for MindWriter, exploration was limited to several interviews and data 
mining of company service records because the concepts were not complicated and the researchers had 
experience in the industry. 

Normally, once the researcher understands the connection between the investigative questions and 
the potential measurement questions, a strategy for the survey is the next logical step. This proceeds to 
getting down to the particulars of instrument design. The following are prominent among the strategic 
concerns: 


1. What type of scale is needed to perform the desired analysis to answer the management 
question? 


2. What communication approach will be used? 
3. Should the questions be structured, unstructured, or some combination? 


4. Should the questioning be undisguised or disguised? If the latter, to what degree? 


Technology has also affected the survey development process, not just the method of the survey’s 
delivery. Today’s software, hardware, and Internet and intranet infrastructures allow researchers to 
(1) write questionnaires more quickly by tapping question banks for appropriate, tested questions, 
(2) create visually driven instruments that enhance the process for the participant, (3) use questionnaire 
software that eliminates separate manual data entry, and (4) build questionnaires that save time in data 
analysis.! 


Type of Scale for Desired Analysis 


The analytical procedures available to the researcher are determined by the scale types used in the 
survey. As Exhibit 13-2 clearly shows, it is important to plan the analysis before developing the mea- 
surement questions. Chapter 12 discussed nominal, ordinal, interval, and ratio scales and explained 
how the characteristics of each type influence the analysis (statistical choices and hypothesis testing). 
We demonstrate how to code and extract the data from the instrument, select appropriate descriptive 
measures or tests, and analyze the results in Chapters 15 through 18. In this chapter, we are most inter- 
ested in asking each question in the right way and in the right order to collect the appropriate data for 
the desired analysis. 
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Today, gathering information reaches into many dimensions: email, chat, surveys, phone conversations, blog posts, 
and more. What you do with that information often determines the difference between success and failure. As Verint 
describes it, “[systems] lacking the capability to analyze captured data in a holistic manner, render valuable informa- 
tion useless because it’s hidden and inaccessible-resulting in isolated, cumbersome decision-making.” Verint offers 
an enterprise feedback management approach, combining survey development, deployment and analysis, as well as 
text analytics and speech analytics, which breaks down information silos and shares data with critical stakeholders, 
showcasing actionable results using customizable, interactive dashboards, like the one shown here. verint.com 


Communication Approach 


As discussed in Chapter 10, communication-based research may be conducted by personal interview, 
telephone, mail, computer (intranet and Internet), or some combination of these (called hybrid studies). 
Decisions regarding which method to use as well as where to interact with the participant (at home, at 
a neutral site, at the sponsor’s place of business, etc.) will affect the design of the instrument. In per- 
sonal interviewing and computer surveying, it is possible to use graphics and other questioning tools 
more easily than it is in questioning done by mail or phone. The different delivery mechanisms result 
in different introductions, instructions, instrument layout, and conclusions. For example, researchers 
may use intercept designs, conducting personal interviews with participants at central locations like 
shopping malls, stores, sports stadiums, amusement parks, or county fairs. The intercept study poses 
several instrument challenges. You’ll find tips for intercept questionnaire design on the text website. 

In the MindWriter example, these decisions were easy. The dispersion of participants, the neces- 
sity of a service experience, and budget limitations all dictated a mail survey in which the participant 
received the instrument either at home or at work. Using a telephone survey, which in this instance is 
the only way to follow up with nonparticipants, could, however, be problematic. This is due to memory 
decay caused by the passage of time between return of the laptop and contact with the participant by 
telephone. 

Jason and Sara have several options for the Albany study. Clearly a self-administered study is pos- 
sible, because all the participants are congregating in a centralized location for scheduled surgery. But 
given the importance of some of the information to medical recovery, a survey conducted via personal 
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interview might be an equally valid choice. We need to know the methodology before we design the 
questionnaire, because some measurement scales are difficult to answer without the visual aid of see- 
ing the scale. 


Disguising Objectives and Sponsors 


Another consideration in communication instrument design is whether the purpose of the study should 
be disguised. A disguised question is designed to conceal the question’s true purpose. Some degree of 
disguise is often present in survey questions, especially to shield the study’s sponsor. We disguise the 
sponsor and the objective of a study if the researcher believes that participants will respond differently 
than they would if both or either was known. 

The accepted wisdom among researchers is that they must disguise the study’s objective or sponsor 
in order to obtain unbiased data. The decision about when to use disguised questions within surveys 
may be made easier by identifying four situations where disguising the study objective is or is not 
an issue: 


e Willingly shared, conscious-level information. 
¢ Reluctantly shared, conscious-level information. 
e¢ Knowable, limited-conscious-level information. 


e Subconscious-level information. 


In surveys requesting conscious-level information that should be willingly shared, either disguised 
or undisguised questions may be used, but the situation rarely requires disguised techniques. 


Example: Have you attended the showing of a foreign language film in the last 
six months? 


In the MindWriter study, the questions revealed in Exhibit 13-13 ask for information that the partici- 
pant should know and be willing to provide. 

Sometimes the participant knows the information we seek but is reluctant to share it for a variety 
of reasons. Exhibit 13-3 offers additional insights as to why participants might not be entirely honest. 
When we ask for an opinion on some topic on which participants may hold a socially unacceptable 
view, we often use projective techniques. (See Chapter 7.) In this type of disguised question, the survey 
designer phrases the questions in a hypothetical way or asks how other people in the participant’s expe- 
rience would answer the question. We use projective techniques so that participants will express their 
true feelings and avoid giving stereotyped answers. The assumption is that responses to these questions 
will indirectly reveal the participants’ opinions. 


Example: Have you downloaded copyrighted music from the Internet without paying for 
it? (nonprojective) 


Example: Do you know people who have downloaded copyrighted music from the Internet 
without paying for it? (projective) 


Not all information is at the participant’s conscious level. Given some time—and motivation—the 
participant can express this information. Asking about individual attitudes when participants know 
they hold the attitude but have not explored why they hold the attitude may encourage the use of 
disguised questions. A classic example is a study of government bond buying during World War II? 
A survey sought reasons why, among people with equal ability to buy, some bought more war bonds 
than others. Frequent buyers had been personally solicited to buy bonds, while most infrequent buyers 
had not received personal solicitation. No direct why question to participants could have provided the 
answer to this question because participants did not know they were receiving differing solicitation 
approaches. 


Example: What is it about air travel during stormy weather that attracts you? 
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>Exhibit 13-3 Factors Affecting Respondent Honesty 


Syndrome Description Example 


Pleaser Respondent gives a politically correct or 
assumed correct answer about degree to 
which they revere their elders, respect their 


spouse, etc. 


Desire to help by providing answers they think the 
researchers want to hear, to please or avoid 
offending or being socially stigmatized. 


Falsify ad recall or purchase behavior (didn’t recall 
or didn’t buy) when they actually did. 


Don’t want to think deeply about a subject. 


Disengager 


Respondent who cannot predict with any certainty 
his future behavior. 


Unconscious Participants who are dominated by irrational 
Decision Maker decision making. 


Source: Developed from an article by Jon Puleston, “Honesty of Responses: The 7 Factors at Play,” GreenBook, March 4, 2012, accessed 
March 5, 2012 (http://www.greenbookblog.org/201 2/03/04/honesty-of-responses-the-7-factors-at-play/) 


In assessing buying behavior, we accept that some motivations are subconscious. This is true for 
attitudinal information as well. Seeking insight into the basic motivations underlying attitudes or con- 
sumption practices may or may not require disguised techniques. Projective techniques (such as sen- 
tence completion tests, cartoon or balloon tests, and word association tests) thoroughly disguise the 
study objective, but they are often difficult to interpret. 


Example: Would you say, then, that the comment you just made indicates you would or 
would not be likely to shop at Galaxy Stores? (survey probe during personal 
interview) 


In the MindWriter study, the questions were direct and undisguised, as the specific information 
sought was at the conscious level. The MindWriter questionnaire is Exhibit 13-13, p. 322. Customers 
knew they were evaluating their experience with the service and repair program at MindWriter; thus 
the purpose of the study and its sponsorship were also undisguised. While the sponsor of the Albany 
Clinic study was obvious, any attempt by a survey to reveal psychological factors that might affect 
recovery and satisfaction might need to use disguised questions. The survey would not want to un- 
necessarily upset a patient before or immediately following surgery, because that might in itself affect 
attitude and recovery. 


Preliminary Analysis Plan 


Researchers are concerned with adequate coverage of the topic and with securing the information 
in its most usable form. A good way to test how well the study plan meets those needs is to develop 
“dummy” tables that display the data one expects to secure. Each dummy table is a cross-tabulation 
between two or more variables. For example, in the biennial study of what Americans eat conducted 
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>Exhibit 13-4 Dummy Table for American Eating Habits 


Use of Convenience Foods 


Use Use 
Always Use Frequently Sometimes Rarely Use Never Use 


18-24 


25-34 


35-44 


45-54 


55-64 


65+ 


by Parade magazine,’ we might be interested to know whether age influences the use of convenience 
foods. The dummy table shown in Exhibit 13-4 would match the age ranges of participants with the de- 
gree to which they use convenience foods. The preliminary analysis plan serves as a check on whether 
the planned measurement questions (e.g., the rating scales on use of convenience foods and on age) 
meet the data needs of the research question. This also helps the researcher determine the type of scale 
needed for each question (e.g., ordinal data on frequency of use and on age)—a preliminary step to 
developing measurement questions for investigative questions. 

In the opening vignette, Jason and Sara use the development of a preliminary analysis plan to de- 
termine whether the project could be kept on budget. The number of hours spent on data analysis is a 
major cost of any survey. Too expansive an analysis plan can reveal unnecessary questions. The guid- 
ing principle of survey design is always to ask only what is needed. 


> Phase 2: Constructing and Refining the 
Measurement Questions 


Drafting or selecting questions begins once you develop a complete list of investigative questions and 
decide on the collection processes to be used. The creation of a survey question is not a haphazard or 
arbitrary process. It is exacting and requires paying significant attention to detail and simultaneously 
addressing numerous issues. Whether you create or borrow or license a question, in Phase 2 (see 
Exhibit 13-5) you generate specific measurement questions considering subject content, the wording 
of each question (influenced by the degree of disguise and the need to provide operational definitions 
for constructs and concepts), and response strategy (each producing a different level of data as needed 
for your preliminary analysis plan). In Phase 3 you must address topic and question sequencing. We 
discuss these topics sequentially, although in practice the process is not linear. For this discussion, we 
assume the questions are structured. 

The order, type, and wording of the measurement questions, the introduction, the instructions, the 
transitions, and the closure in a quality questionnaire should accomplish the following: 


e Encourage each participant to provide accurate responses. 

e Encourage each participant to provide an adequate amount of information. 
e Discourage each participant from refusing to answer specific questions. 

e Discourage each participant from early discontinuation of participation. 


e Leave the participant with a positive attitude about survey participation. 
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>Exhibit 13-5 Flowchart for Instrument Design: Phase 2 


Administrative 
Questions 
|e | ID 
Interviewer ID == 
teats, | Location 
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Question Categories and Structure 


Questionnaires and interview schedules (an alternative term for the questionnaires used in personal 
interviews) can range from those that have a great deal of structure to those that are essentially unstruc- 
tured. Questionnaires contain three categories of measurement questions: 


Pretest Individual 
Questions 


e Administrative questions. 
¢ Classification questions. 
e Target questions (structured or unstructured). 


Administrative questions identify the participant, interviewer, interview location, and conditions. 
These questions are rarely asked of the participant but are necessary for studying patterns within 
the data and identify possible error sources. Classification questions usually cover sociological- 
demographic variables that allow participants’ answers to be grouped so that patterns are revealed 
and can be studied. These questions usually appear at the end of a survey (except for those used as 
filters or screens, questions that determine whether a participant has the requisite level of knowledge 
to participate). Target questions address the investigative questions of a specific study. These are 
grouped by topic in the survey. Target questions may be structured (they present the participants 
with a fixed set of choices; often called closed questions) or unstructured (they do not limit re- 
sponses but do provide a frame of reference for participants’ answers; sometimes referred to as 
open-ended questions). 

In the Albany Clinic study, some questions will need to be unstructured because anticipating medi- 
cations and health history for a wide variety of individuals would be a gargantuan task for a researcher 
and would take up far too much paper space. 


Question Content 


Question content is first and foremost dictated by the investigative questions guiding the study. From 
these questions, questionnaire designers craft or borrow the target and classification questions that will 
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“AS researchers, we need to be sensitive to the unique chal- 
lenges respondents face when completing surveys on mo- 
bile devices,” shared Kristin Luck, CEO of Decipher. “Small 
screens, inflexible device-specific user inout methods, and 
potentially slow data transfer speeds all combine to make 
the survey completion process more difficult than on a typi- 
cal computer. Couple those hindrances with reduced atten- 
tion spans and a lower frustration threshold and it’s clear that, 
as researchers, we must be proactive in the design of both 
the questionnaire and user-interface in order to accommodate 
mobile respondents and provide them with an excellent survey 
experience.” 

Decipher researchers follow key guidelines when designing 
surveys for mobile devices like smart phones and tablets. 


e Ask 10 or fewer questions 
e Minimize page refreshes—longer wait times reduce 
participation. 
e Ask few questions per page—many mobile devices 
have limited memory. 


e Use simple question modes—to minimize scrolling 

e Keep question and answer text short—due to smaller 
screens. 

e If unavoidable, limit scrolling to one dimension (vertical 
is better than horizontal). 

e Use single-response or multiple-response radio button 
or checkbox questions rather than multidimension grid 
questions. 

e Limit open-end questions—to minimize typing. 

e Keep answer options to a short list. 

e For necessary longer answer-list options, use drop- 
down box (but limit these as they require more clicks to 
answer). 


e Minimize all non-essential content 
e If used, limit logos to the first or last survey page. 
e Limit privacy policy to first or last survey page. 


Please indicate your gender: 


select one 


Male 
Female 
Which of the following age groups do you fall into? 


select one 


Under 18 
18-29 
30-54 


55 or over 


Continue » 


10 of 24 


e Debate use of progress bar—it may encourage 
completion but also may require scrolling. 


e Minimize distraction 
e Use simple, high-contrast color schemes — phones 
have limited color palettes. 
e Minimize JavaScript due to bandwidth concerns. 
e Eliminate Flash on surveys—due to incompatibility with 
iPhone. 


Luck is passionate about making sure that researchers recog- 
nize the special requirements of designing for mobile as mobile 
surveys grow in use and projected use, S shares her expertise at 
conferences worldwide. www.decipherinc.com 


be asked of participants. Four questions, covering numerous issues, guide the instrument designer in 
selecting appropriate question content: 


e Should this question be asked (does it match the study objective)? 


¢ Is the question of proper scope and coverage? 


e Can the participant adequately answer this question as asked? 


e Will the participant willingly answer this question as asked? 
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Exhibit 13-6 summarizes these issues related to constructing and refining measurement ques- 
tions that are described here. More detail is provided in Appendix 13a: Crafting Effective Measure- 
ment Questions, available from the text’s Online Learning Center. 


Question Wording 


It is frustrating when people misunderstand a question that has been painstakingly written. This prob- 
lem is partially due to the lack of a shared vocabulary. The difficulty of understanding long and com- 
plex sentences or involved phraseology aggravates the problem further. Our dilemma arises from the 
requirements of question design (the need to be explicit, to present alternatives, and to explain mean- 
ings). All contribute to longer and more involved sentences.* 

The difficulties caused by question wording exceed most other sources of distortion in surveys. 
They have led one social scientist to conclude: 


To many who worked in the Research Branch it soon became evident that error or bias attributable to sampling and to 
methods of questionnaire administration were relatively small as compared with other types of variations — especially 
variation attributable to different ways of wording questions.® 


Although it is impossible to say which wording of a question is best, we can point out several areas that 
cause participant confusion and measurement error. The diligent question designer will put a survey 
question through many revisions before it satisfies these criteria: 


e Is the question stated in terms of a shared vocabulary? 

¢ Does the question contain vocabulary with a single meaning? 

¢ Does the question contain unsupported or misleading assumptions? 
¢ Does the question contain biased wording? 

e Is the question correctly personalized? 

e Are adequate alternatives presented within the question? 


In the vignette, Sara’s study of the prior survey used by the Albany Laser Clinic illustrated several of 
these problems. One question asked participants to identify their “referring physician” and the “physi- 
cian most knowledgeable about your health.” This question was followed by one requesting a single 
phone number. Participants didn’t know which doctor’s phone number was being requested. By offer- 
ing space for only one number, the data collection instrument implied that both parts of the question 
might refer to the same doctor. Further, the questions about past medical history did not offer clear 
directions. One question asked participants about whether they had “had the flu recently,” yet made no 
attempt to define whether recently was within the last 10 days or the last year. Another asked “Are your 
teeth intact?” Prior participants had answered by providing information about whether they wore false 
teeth, had loose teeth, or had broken or chipped teeth—only one of which was of interest to the doctor 
performing surgery. To another question (“Do you have limited motion of your neck?”), all respondents 
answered yes. Sara could only conclude that a talented researcher did not design the clinic’s previously 
used questionnaire. Although the Albany Outpatient Laser Clinic survey did not reveal any leading 
questions, these can inject significant error by implying that one response should be favored over an- 
other. One classic hair care study asked, “How did you like Brand X when it lathered up so nicely?” 
Obviously, the participant was supposed to factor in the richness of the lather in evaluating the shampoo. 

The MindWriter questionnaire (see Exhibit 13-13) simplified the process by using the same re- 
sponse strategy for each factor the participant was asked to evaluate. The study basically asks, “How 
did our CompleteCare service program work for you when you consider each of the following factors?” 
It accomplishes this by setting up the questioning with “Take a moment to tell us how well we’ve 
served you.” Because the sample includes CompleteCare users only, the underlying assumption that 
participants have used the service is acceptable. The language is appropriate for the participant’s likely 
level of education. And the open-ended question used for “comments” adds flexibility to capture any 
unusual circumstances not covered by the structured list. 

Target questions need not be constructed solely of words. Computer-assisted, computer-administered, 
and online surveys and interview schedules, and to a lesser extent printed surveys, often incorporate 
visual images as part of the questioning process. 
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Exhibit 13-6 A Summary of the Major Issues Related to Measurement Questions 


Issue Category Fundamental Issue 


Question Content 


1. Purposeful versus interesting 


2. Incomplete or unfocused 


8. Double-barreled questions 


4. Precision 


5. Time for thought 


6. Participation at the expense of accuracy 


7. Presumed knowledge 


8. Recall and memory decay 


9. Balance (general vs. specific) 


10. Objectivity 


11. Sensitive information 


Question Wording 


12. Shared vocabulary 


13. Unsupported assumption 


14. Frame of reference 


15. Biased wording 


16. Personalization vs. projection 


17. Adequate alternatives 


Response Strategy Choice 


18. Objective of the study 


19. Level of information 


20. Thoroughness of prior thought 


21. Communication skill 


22. Participant motivation 


Does the question ask for data that will be merely interesting or truly useful in 
making a decision? 


Will the question reveal what the decision maker needs to know? 


Does the question ask the participant for too much information? Would the 
desired single response be accurate for all parts of the question? 


Does the question ask precisely what the decision maker needs to know? 


s it reasonable to assume that the participant can frame an answer to the 
question? 


Does the question pressure the participant for a response regardless of knowl- 
edge or experience? 


Does the question assume the participant has knowledge he or she may not 
have? 


Does the question ask the participant for information that relates to thoughts 
or activity too far in the participant’s past to be remembered? 


Does the question ask the participant to generalize or summarize behavior that 
may have no discernable pattern? 


Does the question omit or include information that will bias the participant’s 
response? 


Does the question ask the participant to reveal embarrassing, shameful, or 
ego-related information? 


Does the question use words that have no meaning or a different meaning for the 
participant? 

Does the question assume a prior experience, a precondition, or prior knowledge 
that the participant does not or may not have? 

s the question worded from the participant’s, rather than the researcher's, 
perspective? 

Does the question contain wording that implies the researcher’s desire for the 
participant to respond in one way versus another? 


s it necessary for the participant to reveal personal attitudes and behavior, or 
may the participant project these attitudes and behaviors to someone like him 
or her? 


Does the question provide a mutually exhaustive list of alternatives to encompass 
realistic or likely participant attitudes and behaviors? 


s the question designed to classify or label attitudes, conditions, and behaviors or 
to reveal them? 


Does the participant possess the level of information appropriate for participation 
in the study? 


Has the participant developed an attitude on the issue being asked? 


Does the participant have sufficient command of the language to answer the 
question? 


s the level of motivation sufficient to encourage the participant to give thoughttul, 
revealing answers? 
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>Exhibit 13-7 Internet Survey Response Options 


Free Response/Open Question 
using textbox 


| plan to purchase a MindWriter laptop in the next 3 months. 


© Yes 

Dichotomous Question © No 
using radio buttons 

(may also use pull-down box) 


Paired Comparison 
using radio buttons 
(may also use pull-down box) 


What ONE magazine do you read most often for computing news? 
© PC Magazine 
Multiple Choice, Single © Wired 
Response anal - 
using radio buttons © Bonbuung cee 
(may also use pull-down box © Computing ee d 
or checkbox) © PC Computing 
© Laptop 


Response Strategy 


A third major decision area in question design is the degree and form of structure imposed on the 
participant. The various response strategies offer options that include unstructured response (or 
open-ended response, the free choice of words) and structured response (or closed response, 
specified alternatives provided). Free responses, in turn, range from those in which the partici- 
pants express themselves extensively to those in which participants’ latitude is restricted by space, 
layout, or instructions to choose one word or phrase, as in a fill-in question. Closed responses 
typically are categorized as dichotomous, multiple-choice, checklist, rating, or ranking response 
strategies. 


>Exhibit 13-7 (Cont'd) 


Please select your answer V 


PC Magazine 

Wired 

Computing Magazine 
Computing World 

PC Computing 
Laptop 


Checklist 
using checkbox 
(may also use radio buttons) 


Fast reliable repair service 


Service at my location 
Maintenance by the manufacturer 
Knowledgeable technicians 
Notification of upgrades 


Ranking Question 
using pull-down box 
(may also use textboxes, 
in which ranks are entered) 
[This question asks for 
a limited ranking of 
only three of the 
listed elements.] 
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Multiple Choice, Single 
Response 
using pull-down box 


Which of the following computing magazines did you look at in the last 30 days? 
PC Magazine 
Wired 

(_| Computing Magazine 

(_] Computing World 

(_] PC Computing 

(_] Laptop 


Rating Grid 
(may also use checkboxes) 
Requires a single response per line. 
The longer the list, the more likely 
the participant must scroll. 


From the list below, please choose the three most important service options when 
choosing your next laptop. 


Fast reliable repair service 


Service at my location 


Maintenance by the manufacturer 


Knowledgeable technicians 


—v 
- 
3 

_— 


Notification of upgrades 


Several situational factors affect the decision of whether to use open-ended or closed questions.’ The 
decision is also affected by the degree to which these factors are known to the interviewer. The factors are: 


¢ Objectives of the study. 


e Participant’s level of information about the topic. 


¢ Degree to which participant has thought through the topic. 


e Ease with which participant communicates. 


e Participant’s motivation level to share information. 


All of the strategies that are described in this section are available for use on Web questionnaires. How- 
ever, with the Web survey you are faced with slightly different layout options for response, as noted in 
Exhibit 13-7. For the multiple-choice or dichotomous response strategies, the designer chooses between 
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radio buttons and drop-down boxes. For the checklist or multiple response strategy, the designer must 
use the checkbox. For rating scales, designers may use pop-up windows that contain the scale and 
instructions, but the response option is usually the radio button. For ranking questions, designers use 
radio buttons, drop-down boxes, and textboxes. For the free response question, the designer chooses 
either the one-line textbox or the scrolled textbox. Web surveys and other computer-assisted surveys 
can return participants to a given question or prompt them to complete a response when they click the 
“submit” button; this is especially valuable for checklists, rating scales, and ranking questions. You 
may wish to review Exhibits 12-3 and 12-10. These provide other question samples. 


Free-Response Question 


Free-response questions, also known as open-ended questions, ask the participant a question and 
either the interviewer pauses for the answer (which is unaided) or the participant records his or her 
ideas in his or her own words in the space provided on a questionnaire. Survey researchers usually try 
to reduce the number of such questions because they pose significant problems in interpretation and are 
costly in terms of data analysis. 


Dichotomous Question 


A topic may present clearly dichotomous choices: something is a fact or it is not; a participant can 
either recall or not recall information; a participant attended or didn’t attend an event. Dichotomous 
questions suggest opposing responses, but this is not always the case. One response may be so unlikely 
that it would be better to adopt the middle-ground alternative as one of the two choices. For example, 
if we ask participants whether a product is underpriced or overpriced, we are not likely to get many 
selections of the former choice. The better alternatives to present to the participant might be “fairly 
priced” or “overpriced.” 

In many two-way questions, there are potential alternatives beyond the stated two alternatives. If 
the participant cannot accept either alternative in a dichotomous question, he or she may convert the 
question to a multiple-choice or rating question by writing in his or her desired alternative. For ex- 
ample, the participant may prefer an alternative such as “don’t know” to a yes-no question or prefer 
“no opinion” when faced with a favor-oppose option. In other cases, when there are two opposing 
or complementary choices, the participant may prefer a qualified choice (“yes, if X doesn’t occur,” 
or “sometimes yes and sometimes no,” or “about the same’’). Thus, two-way questions may become 
multiple-choice or rating questions, and these additional responses should be reflected in your revised 
analysis plan. Dichotomous questions generate nominal data. 


Multiple-Choice Question 


Multiple-choice questions are appropriate when there are more than two alternatives or when we 
seek gradations of preference, interest, or agreement; the latter situation also calls for rating questions. 
Although such questions offer more than one alternative answer, they request that the participant make 
a single choice. Multiple-choice questions can be efficient, but they also present unique design and 
analysis problems. 

One type of problem occurs when one or more responses have not been anticipated. Assume we 
ask whether retail mall security and safety rules should be determined by the (1) store managers, 
(2) sales associates who work at the mall, (3) federal government, or (4) state government. The union 
has not been mentioned in the alternatives. Many participants might combine this alternative with 
“sales associates,” but others will view “unions” as a distinct alternative. Exploration prior to drafting 
the measurement question attempts to identify the most likely choices. 

A second problem occurs when the list of choices is not exhaustive. Participants may want to give 
an answer that is not offered as an alternative. This may occur when the desired response is one that 
combines two or more of the listed individual alternatives. Many participants may believe the store 
management and the sales associates acting jointly should set store safety rules, but the question does 
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Travel Issues and Transportation Security Administration's 
Registered Traveler Program (RTP) 
(n = 1580) 


Percent unaware of RTP 


Percent who say biggest 
travel problem is long lines 
at airports 


Percent uninterested in RTP 


Percent with privacy concerns 


Percent of travelers who 
would consider enrolling if 
company paid registration fee 


Percent of frequent travelers 
who would consider enrolling 


>picprofile 


Organizations use questionnaires to measure all sorts of activities and attitudes. Kraft used an in-magazine questionnaire 
to measure whether its Food and Family magazine readers wanted tip-in stickers to mark favorite recipe pages. The Kroger 
Company, Applebee’s restaurants, and Kohl’s Department Store use automated phone surveys to measure customer satis- 
faction. Deloitte & Touche USA LLP used an online questionnaire to measure understanding of the Registered Traveler Pro- 
gram for the Transportation Security Administration. This program promises that registered travelers will not have to contend 
with long lines at terminal entrances. Some findings from this survey are noted in the accompanying graph. www.tsa.gov; 
www.deloitte.com 


not include this response. When the researcher tries to provide for all possible options, choosing from 
the list of alternatives can become exhausting. We guard against this by discovering the major choices 
through exploration and pretesting (discussed in detail in Appendix 13b, available from the text Online 
Learning Center). We may also add the category “Other (please specify)” as a safeguard to provide 
the participant with an acceptable alternative for all other options. In our analysis of responses to a 
pretested, self-administered questionnaire, we may create a combination alternative. 

Yet another problem occurs when the participant divides the question of store safety into several 
questions, each with different alternatives. Some participants may believe rules dealing with air qual- 
ity in stores should be set by a federal agency while those dealing with aisle obstructions or displays 
should be set by store management and union representatives. Still others may want store management 
in conjunction with a sales associate committee to make rules. To address this problem, the instrument 
designer would need to divide the question. Pretesting should reveal if a multiple-choice question is 
really a double-barreled question. 

Another challenge in alternative selection occurs when the choices are not mutually exclusive (the 
participant thinks two or more responses overlap). In a multiple-choice question that asks students, 
“Which one of the following factors was most influential in your decision to attend Metro U?” these 
response alternatives might be listed: 


1. Good academic reputation. 

. Specific program of study desired. 
. Enjoyable campus life. 

. Many friends from home attend. 


. High quality of the faculty. 


nn FW WN 


. Opportunity to play collegiate-level sports. 
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Some participants might view items | and 5 as overlapping, and some may see items 3 and 4 in the 
same way. 

It is also important to seek a fair balance in choices when a participant’s position on an issue is 
unknown. One study showed that an off-balance presentation of alternatives biases the results in favor 
of the more heavily offered side.* If four gradations of alternatives are on one side of an issue and two 
are offered reflecting the other side, responses will tend to be biased toward the better-represented side. 
However, researchers may have a valid reason for using an unbalanced array of alternatives. They may 
be trying to determine the degree of positive (or negative) response, already knowing which side of an 
issue most participants will choose based on the selection criteria for participation. 

It is necessary in multiple-choice questions to present reasonable alternatives—particularly when the 
choices are numbers or identifications. If we ask, “Which of the following numbers is closest to the num- 
ber of students enrolled in American colleges and universities today?” these choices might be presented: 


1. 75,000 
2. 750,000 
3. 7,500,000 
4. 25,000,000 
5. 75,000,000 


It should be obvious to most participants that at least three of these choices are not reasonable, given 
general knowledge about the population of the United States and about the colleges and universities 
in their hometowns. (The estimated 2006 U.S. population is 298.4° million based on the 2000 census 
of 281.4 million. The Ohio State University has more than 59,000" students.) 

The order in which choices are given can also be a problem. Numeric alternatives are normally 
presented in order of magnitude. This practice introduces a bias. The participant assumes that if there 
is a list of five numbers, the correct answer will lie somewhere in the middle of the group. Researchers 
are assumed to add a couple of incorrect numbers on each side of the correct one. To counteract this 
tendency to choose the central position, put the correct number at an extreme position more often when 
you design a multiple-choice question. 

Order bias with nonnumeric response categories often leads the participant to choose the first alter- 
native (primacy effect) or the last alternative (recency effect) over the middle ones. Primacy effect 
dominates in visual surveys—self-administered via Web or mail—while recency effect dominates in 
oral surveys—phone and personal interview surveys.'! Using the split-ballot technique can counteract 
this bias: Different segments of the sample are presented alternatives in different orders. To implement 
this strategy in face-to-face interviews, the researcher would list the alternatives on a card to be handed 
to the participant when the question is asked. Cards with different choice orders can be alternated to 
ensure positional balance. The researcher would leave the choices unnumbered on the card so that the 
participant replies by giving the response category itself rather than its identifying number. It is a good 
practice to use cards like this any time there are four or more choice alternatives. This saves the inter- 
viewer reading time and ensures a more valid answer by keeping the full range of choices in front of 
the participant. With computer-assisted surveying, the software can be programmed to rotate the order 
of the alternatives so that each participant receives the alternatives in randomized order (for nonordered 
scales) or in reverse order (for ordered scales). 

In most multiple-choice questions, there is also a problem of ensuring that the choices represent a 
one-dimensional scale—that is, the alternatives to a given question should represent different aspects 
of the same conceptual dimension. In the college selection example, the list included features associ- 
ated with a college that might be attractive to a student. This list, although not exhaustive, illustrated 
aspects of the concept “college attractiveness factors within the control of the college.” The list did 
not mention other factors that might affect a school attendance decision. Parents and peer advice, local 
alumni efforts, and one’s high school adviser may influence the decision, but these represent a different 
conceptual dimension of “college attractiveness factors” —those not within the control of the college. 

Multiple-choice questions usually generate nominal data. When the choices are numeric alternatives, 
this response structure may produce at least interval and sometimes ratio data. When the choices rep- 
resent ordered but unequal, numerical ranges (e.g., a question on family income: <$20,000; $20,000- 
$100,000; >$100,000) or a verbal rating scale (e.g., a question on how you prefer your steak prepared: 
well done, medium well, medium rare, or rare), the multiple-choice question generates ordinal data. 
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@® Invoke Solutions - Welcome - Microsoft Internet Explorer 


File Edit Yiew Favorites Tools Help 


Qa ~ © - |x) (2) GE) seach sly ravortes @Preda €) ~~ B& 


tres | nvo KO scinaa 


SimantaBals 


Smart Bar. The at-fi bar that helps you feel 
satisfied between me sre, e and fast ht loss peter, 
grams. This great ‘Which of the following age groups best represents your age? 
rams of hung ti 
Smart Bar t s than 17 gram arbs (under 
f sugar). And th Jart B. with the vitamins 
and minerals necessary J ind light snack. 3, 40-49 
Smart Bar, T E 4, 50-55 


1, 22-29 


2, 30-39 


Peter: 
How appealing to you is this product? 


1, Very appealing 
2. Somewhat appealing 
3, Neither appealing nor unappealing 


te) 
le) 
(2) 
1e) 


4, Somewhat unappealing 


© Yer iinannedine 


Peter: 
DM iaCi elas Amount/Serving %DV*| Amount/Serving %DV* 


What appeals to you most about this product? 
| Total Fat Og 0% | Cholesterol Omg 0% 
Serv. Size 1 bar (379) | 


Dietary Fiber 2g 8% | Sat Fat Og 0% My Answer: 
Total Calories 80 ©—-|_S0dlum340mg_ 15% | Sugers 6g The fact that it contains soy and has no saturated fats, (Send) 


| Total Carbs 169 Protein 8g ly (er) 


"Percent Daily Values are based on @ 2,000 calorie diet Cancel] 
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One option that lets you combine the best of group interview methodology with the power of population-representative survey 
methodology is Invoke Solutions’ Invoke Engage. With Invoke Engage, a moderator coordinates responses of up to 200 partici- 
pants in a single live session that lasts between 60 and 90 minutes. Moderators ask prescreened, recruited participants closed 
questions. These can include not only text (e.g., new safety policy) but also visual (e.g., Web design options) and full-motion 
video (e.g., training segment) stimuli. Participants respond in ways similar to an online questionnaire. Interspersed with these 
quantitative measures are opportunities to dig deeply with open-ended questions. These questions are designed to reveal 
participants’ thinking and motivations. Participants keyboard their responses, which are electronically sorted into categories. 
At the moderator’s initiation, participants might see a small, randomly generated sample of other participants’ responses and 
be asked to agree or disagree with these responses. Monitoring sponsors obtain real-time frequency tallies and verbatim 
feedback, as well as end-of-session transcripts. Within a few days sponsors receive content-analyzed verbatims and detailed 
statistical analysis of closed-question data, along with Invoke Solutions’ recommendations on the hypothesis that drove the 
research. www.invoke.com 


Checklist 


When you want a participant to give multiple responses to a single question, you will ask the ques- 
tion in one of three ways: the checklist, rating, or ranking strategy. If relative order is not important, 
the checklist is the logical choice. Questions like “Which of the following factors encouraged you to 
apply to Metro U? (Check all that apply)” force the participant to exercise a dichotomous response 
(yes, encouraged; no, didn’t encourage) to each factor presented. Of course, you could have asked for 
the same information with a series of dichotomous selection questions, one for each individual factor, 
but this would have been both time- and space-consuming. Checklists are more efficient. Checklists 
generate nominal data. 


Rating Question 


Rating questions ask the participant to position each factor on a companion scale, either verbal, nu- 
meric, or graphic. “Each of the following factors has been shown to have some influence on a student’s 
choice to apply to Metro U. Using your own experience, for each factor please tell us whether the factor 
was ‘strongly influential,’ ‘somewhat influential,’ or ‘not at all influential.’” Generally, rating-scale 
structures generate ordinal data; some carefully crafted scales generate interval data. 


312 


>part Ill. The Sources and Collection of Data 


It is important to remember that the researcher should represent only one response dimension in 
rating-scale response options. Otherwise, effectively, you present the participant with a double-barreled 
question with insufficient choices to reply to both aspects. 


Example A: — How likely are you to enroll at Metro University? 
(Responses with more than one dimension, ordinal scale) 
(a) extremely likely to enroll 
(b) somewhat likely to enroll 
(c) not likely to apply 
(d) will not apply 


Example B: How likely are you to enroll at Metro University? 
(Responses within one dimension, interval scale) 
(a) extremely likely to enroll 
(b) somewhat likely to enroll 
(c) neither likely nor unlikely to enroll 
(d) somewhat unlikely to enroll 
(e) extremely unlikely to enroll 


Ranking Question 


When relative order of the alternatives is important, the ranking question is ideal. “Please rank-order 
your top three factors from the following list based on its influence in encouraging you to apply to 
Metro U. Use | to indicate the most encouraging factor, 2 the next most encouraging factor, etc.” The 
checklist strategy would provide the three factors of influence, but we would have no way of knowing 
the importance the participant places on each factor. Even in a personal interview, the order in which 
the factors are mentioned is not a guarantee of influence. Ranking as a response strategy solves this 
problem. 

One concern surfaces with ranking activities. How many presented factors should be ranked? If you 
listed the 15 brands of potato chips sold in a given market, would you have the participant rank all 15 in 
order of preference? In most instances it is helpful to remind yourself that while participants may have 
been selected for a given study due to their experience or likelihood of having desired information, 
this does not mean that they have knowledge of all conceivable aspects of an issue, but only of some. 
It is always better to have participants rank only those elements with which they are familiar. For this 
reason, ranking questions might appropriately follow a checklist question that identifies the objects of 
familiarity. If you want motivation to remain strong, avoid asking a participant to rank more than seven 
items even if your list is longer. Ranking generates ordinal data. 

All types of response strategies have their advantages and disadvantages. Several different strategies 
are often found in the same questionnaire, and the situational factors mentioned earlier are the major 
guides in this matter. There is a tendency, however, to use closed questions instead of the more flexible 
open-ended type. Exhibit 13-8 summarizes some important considerations in choosing between the 
various response strategies. 


Sources of Existing Questions 


The tools of data collection should be adapted to the problem, not the reverse. Thus, the focus of this 
chapter has been on crafting an instrument to answer specific investigative questions. But inventing, re- 
fining, and pretesting questions demands considerable time and effort. For some topics, a careful review 
of the related literature and an examination of existing instrument sourcebooks can shorten this process. 
Increasingly, companies that specialize in survey research maintain a question bank of pretested ques- 
tions. In the opening vignette, Sara was accessing Henry and Associates’ question bank. 
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Exhibit 13-8 Summary of Scale Types 
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Type Restrictions Scale Items Data Type 
Rating Scales 
Simple Category Needs mutually exclusive choices. One or more Nominal 
Scale 
Multiple Choice Needs mutually exclusive choices; may use Many Nominal 
Single-Response exhaustive list or “other.” 
Scale 
Multiple Choice Needs mutually exclusive choices; needs exhaus- Nominal 
Multiple- tive list or “other.” 
Response Scale 
(checklist) 
Likert Scale Needs definitive positive or negative statements One or more Interval 
with which to agree/disagree. 
Likert-type Scale Needs definitive positive or negative statements One or more Ordinal or interval 
with which to agree/disagree. 
Semantic Differen- Needs words that are opposites to anchor the One or more Interval 
tial Scale graphic space. 
Numerical Scale Needs concepts with standardized or defined One or many Ordinal or 
meanings; needs numbers to anchor the interval 


Multiple Rating List 
Scale 


Fixed (Constant) 
Sum Scale 


Stapel Scale 


Graphic Rating 


Scale positive or negative anchors; score is a mea- ratio 
surement of graphical space from one anchor. 
Ranking Scales 

Paired Comparison Number is controlled by participant’s stamina and Up to 10 Ordinal 
Scale interest. 

Forced Ranking Needs mutually exclusive choices. Up to 10 Ordinal 
Scale 

Comparative Scale Can use verbal or graphical scale. Up to 10 Ordinal 


end-points or points along the scale; score is 
a measurement of graphical space from one 
anchor. 


Needs words that are opposites to anchor the 
end-points on the verbal scale. 


Participant needs ability to calculate total to some 
fixed number, often 100. 


Needs verbal labels that are operationally defined 
or standard. 


Needs visual images that can be interpreted as 


Up to 10 


Two or more 


One or more 


One or more 


Ordinal or interval 


Ratio 


Ordinal or interval 


Ordinal, interval, or 


A review of literature will reveal instruments used in similar studies that may be obtained by writ- 


ing to the researchers or, if copyrighted, may be purchased through a clearinghouse. Instruments also 
are available through compilations and sourcebooks. While these tend to be oriented to social science 
applications, they are a rich source of ideas for tailoring questions to meet a manager’s needs. Several 
compilations are recommended; we have suggested them in Exhibit 13-9.!* 

Borrowing items from existing sources is not without risk. It is quite difficult to generalize the reli- 
ability and validity of selected questionnaire items or portions of a questionnaire that have been taken 
out of the original context. Researchers whose questions or instruments you borrow may not have 
reported sampling and testing procedures needed to judge the quality of the measurement scale. Just 
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Exhibit 13-9 Sources of Questions 


The Sources and Collection of Data 


Printed Sources 


Source 


Author(s) 


William Bearden, R. Netemeyer, and 
Kelly L. Haws 


Alec Gallup and Frank Newport, eds. 


John P. Robinson, Philip R. Shaver, 
and Lawrence S. Wrightsman 


John Robinson, Phillip R. Shaver, and 
L. Wrightsman 


Alec M. Gallup 


Gordon Bruner, Paul Hensel, and 
Karen E. James 


Elizabeth H. Hastings and Philip K. 
Hastings, eds. 


Elizabeth Martin, Diana McDuffee, and 
Stanley Presser 


Philip E. Converse, Jean D. Dotson, 
Wendy J. Hoag, and William H. 
McGee Ill, eds. 


Philip K. Hastings and Jessie C. 
Southwick, eds. 


National Opinion Research Center 


John P. Robinson 


Handbook of Marketing Scales: Multi-Item 
Measures for Marketing and Consumer 
Behavior Research 


The Gallup Poll Cumulative Index: Public 
Opinion, 1998-2007 


Measures of Personality and Social- 
Psychological Attitudes 


Measures of Political Attitudes 


The Gallup Poll: Public Opinion 2010 


Marketing Scales Handbook, 
Volume IV: Consumer Behavior 


Index to International Public Opinion, 
1996-1997 


Sourcebook of Harris National Surveys: 
Repeated Questions 1963-1976 


American Social Attitudes Data 
Sourcebook, 1947-1978 


Survey Data for Trend Analysis: An 
Index to Repeated Questions in the 
U.S. National Surveys Held by the 
Roper Public Opinion Research 
Center 


General Social Surveys 1972-2000: 
Cumulative Code Book 


Measures of Occupational Attitudes and 
Occupational Characteristics 


London: Sage Publications, Inc., 2010 


Lanham, Maryland: Rowman & Littlefield 
Publishers, Inc., 2008 


San Diego, CA: Academic Press, 1990, 
1999 


San Diego, CA: Academic Press, 1990, 
1999 


Lanham, Maryland: Rowman & Littlefield 
Publishers, Inc., 2011 


South-Western Educational Pub, 2005 


Westport, CT: Greenwood Publishing 
Group, 1998 


Chapel Hill, NC: Institute for Research in 
Social Science, 1981 


Cambridge, MA: Harvard University 
Press, 1980 


Williamsburg, MA: Roper Public Opinion 
Center, 1975 


Ann Arbor, MI: ICPSR, 2000 


Ann Arbor, MI: Institute for Social 
Research, University of Michigan, 
alive 


Interuniversity Consortium for Political 
and Social Research (general social 
survey) 


iPoll (contains more than 500,000 
questions in its searchable database) 


Survey Research Laboratory, Florida 
State University 


The Odum Institute (houses the Louis 
Harris Opinion Polls) 


Kaiser Family Foundation Health Poll 
Search 


Polling the Nations (more than 14,000 
surveys) 


www.icpsr.umich.edu 


Wwww.ropercenter.uconn.edu 


http://survey.coss.fsu.edu/index.htm 


http://www. irss.unc.edu/odum/home2.jsp 


www.kff.org/kaiserpolls/healthpoll.cfm 


www.orspub.com 
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because Jason has a satisfaction scale in the question bank used for the CardioQuest survey does not 
mean the question will be appropriate for the Albany Outpatient Laser Clinic. Sara would need to know 
the intended purpose of the CardioQuest study and the time of construction, as well as the results of 
pretesting, to determine the reliability and validity of its use in the Albany study. Even then she would 
be wise to pretest the question in the context of her Albany survey. 

Language, phrasing, and idioms can also pose problems. Questions tend to age or become out- 
dated and may not appear (or sound) as relevant to the participant as freshly worded questions. 
Integrating previously used and customized questions is problematic. Often adjacent questions in 
one questionnaire are relied on to carry context. If you select one question from a contextual series, 
the borrowed question is left without its necessary meaning.'* Whether an instrument is constructed 
with designed questions or adapted with questions borrowed or licensed from others, pretesting is 
expected. 


> Phase 3: Drafting and Refining 
the Instrument 


As depicted in Exhibit 13-10, Phase 3 of instrument design—drafting and refinement—is a multistep 
process: 


1. Develop the participant-screening process (done especially with personal or phone surveys, 
but also with early notification procedures with e-mail and Web surveys), along with the 
introduction. 


>Exhibit 13-10 Flowchart for Instrument Design: Phase 3 


Revise 


Instrument Pretest 
Design Questionnaire 


Administrative Questions | 
Introduction and Screen with 
Participant Instructions e Use filter questions to 


screen prospect 


e Establish rapport with 
buffer questions 


ee | ¢ Build interest with early 


target questions 


e Sequence questions from 


Transition to general to specific 
Classification Questions 
e Include skip directions to 
Conclusion with Instrument 
Disposition Instructions 


facilitate sequencing 
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>Exhibit 13-11 Sample Components of Communication Instruments 


Component Example 


Introduction 
a. Phone/personal interview Good evening. May | please speak with (name of participant)? 

Mr. (participant’s last name), I’m (your name), calling on behalf of MindWriter Corporation. You 
recently had your MindWriter laptop serviced at our CompleteCare Center. Could you take five 
minutes to tell us what you thought of the service provided by the Center? 

b. Online (often delivered via You've recently had your MindWriter laptop serviced at our CompleteCare Center. Could you take five 
email) minutes to tell us what you thought of the service provided by the Center? Just click the link below. 


Instructions for... 


a. Terminating (following filter Phone: |’m sorry, today we are only talking with individuals who eat cereal at least three days per 
or screen question) week, but thank you for speaking with me. (Pause for participant reply.) Good-bye. 
Online: You do not qualify for this particular study. Click below to see other studies for which you 
might qualify. 


c. Skip directions (between 
questions or groups of 
questions...paper or 
phone) 


3. Did you purchase boxed cereal in the last 7 days? 
LI Yes 
(4 No (skip to question 7) 


Conclusion 
a. Phone or personal 
interview 


That’s my last question. Your insights and the ideas of other valuable customers will help us to 
make the CompleteCare program the best it can be. Thank you for talking with us this evening. 
(Pause for participant reply.) Good evening. 


2. Arrange the measurement question sequence: 
a. Identify groups of target questions by topic. 
b. Establish a logical sequence for the question groups and questions within groups. 
c. Develop transitions between these question groups. 


3. Prepare and insert instructions for the interviewer—including termination instructions, skip 
directions, and probes for the participants. 


4. Create and insert a conclusion, including a survey disposition statement. 


5. Pretest specific questions and the instrument as a whole. 


Participant Screening and Introduction 


The introduction must supply the sample unit with the motivation to participate in the study. It must 
reveal enough about the forthcoming questions, usually by revealing some or all of the topics to be 
covered, for participants to judge their interest level and their ability to provide the desired information. 
In any communication study, the introduction also reveals the amount of time participation is likely to 
take. The introduction also reveals the research organization or sponsor (unless the study is disguised) 
and possibly the objective of the study. In personal or phone interviews as well as in e-mail and Web 
surveys, the introduction usually contains one or more screen questions or filter questions to determine 
if the potential participant has the knowledge or experience necessary to participate in the study. At a 
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2. Which of the following attributes do you like about the automobile you just saw? (Select all that apply.) 
(Yj Overall appeal 

7] Headroom 

(_] Design 

[_] Color 

\Y| Height from the ground 

(_] Other j 


(_] None of the above 


3. For those items that you selected, how important is each? (Provide one answer for each attribute.) 


Neither 
important 
Extremely nor not Not at all Don't 
important important important know 
a) Overall appeal O O O O Oo e) 
b) Height from the ground O O O Oo O Oo 
c) Headroom O ©) Oo Oo Oo e) 


>picprofile 


One of the attractions of using a Web survey is the ease with which participants follow branching questions immediately customized to 
their response patterns. In this survey, participants were shown several pictures of a prototype vehicle. Those who responded to ques- 
tion 2 by selecting one or more of the attributes in the checklist question were sequenced to a version of question 3 that related only to 
their particular responses to question 2. Note also that in question 3 the researcher chose not to force an answer, allowing the partici- 
pant to indicate he or she had no opinion (“Don’t know’) on the issue of level of importance. 


minimum, a phone or personal interviewer will provide his or her first name to help establish critical 
rapport with the potential participant. Additionally, more than two-thirds of phone surveys contain a 
statement that the interviewer is “not selling anything.”’'* Exhibit 13-11 provides a sample introduction 
and other components of a telephone study of nonparticipants to a self-administered mail survey. 


Measurement Question Sequencing 


The design of survey questions is influenced by the need to relate each question to the others in the 
instrument. Often the content of one question (called a branched question) assumes other questions 
have been asked and answered. The psychological order of the questions is also important; ques- 
tion sequence can encourage or discourage commitment and promote or hinder the development of 
researcher-participant rapport. 

The basic principle used to guide sequence decisions is this: the nature and needs of the participant 
must determine the sequence of questions and the organization of the interview schedule. Four guide- 
lines are suggested to implement this principle: 


1. The question process must quickly awaken interest and motivate the participant to participate 
in the interview. Put the more interesting topical target questions early. Leave classification 
questions (e.g., age, family size, income) not used as filters or screens to the end of the survey. 


2. The participant should not be confronted by early requests for information that might be 
considered personal or ego-threatening. Put questions that might influence the participant to 
discontinue or terminate the questioning process near the end. 


3. The questioning process should begin with simple items and then move to the more complex, 
as well as move from general items to the more specific. Put taxing and challenging questions 
later in the questioning process. 


4. Changes in the frame of reference should be small and should be clearly pointed out. Use 
transition statements between different topics of the target question set. 
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Maximum Online Survey Length Prior to Abandonment 


More than 20 minutes, 
13.3% 


16-20 minutes, 9.0% 5 minutes or less, 


33.9% 


11-15 minutes, 15.7% 


15 minutes or less, 77.6% 


6-10 minutes, 28% 


> pic orofi la 10 minutes or less, 61.9% 


As marketing resistance rises and survey cooperation declines, survey length is of increasing concern. InsightExpress stud- 
ied the Web survey process and revealed that people taking Web surveys prefer shorter to longer surveys, consistent with 
what we know about phone and intercept survey participants. While 77 percent were likely to complete a survey that took 
15 minutes or less, almost one in three participants needed a survey to be 5 minutes or less for full completion. As participat- 
ing in online surveys loses its novelty, prospective participants are likely to become even more reluctant to give significant 
time to the survey process. Therefore, it is critical that researchers ask only what is necessary. www.insightexpress.com 


Awaken Interest and Motivation 


We awaken interest and stimulate motivation to participate by choosing or designing questions that are 
attention-getting and not controversial. If the questions have human-interest value, so much the better. 
It is possible that the early questions will contribute valuable data to the major study objective, but their 
major task is to overcome the motivational barrier. 


Sensitive and Ego-Involving Information 


Regarding the introduction of sensitive information too early in the process, two forms of this error 
are common. Most studies need to ask for personal classification information about participants. Par- 
ticipants normally will provide these data, but the request should be made at the end of the survey. If 
made at the start of the survey, it often causes participants to feel threatened, dampening their interest 
and motivation to continue. It is also dangerous to ask any question at the start that is too personal. 
For example, participants in one survey were asked whether they suffered from insomnia. When the 
question was asked immediately after the interviewer’ s introductory remarks, about 12 percent of those 
interviewed admitted to having insomnia. When a matched sample was asked the same question after 
two buffer questions (neutral questions designed chiefly to establish rapport with the participant), 
23 percent admitted suffering from insomnia." 


Simple to Complex 


Deferring complex questions or simple questions that require much thought can help reduce the num- 
ber of “don’t know” responses that are so prevalent early in interviews. 
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Does Cupid Deserve a Place in the Office Cubicle? 


As a manager, should you encourage or discour- 
age office romance? Spherion Inc., a leading re- 
cruiting and staffing company, recently sponsored 
its latest Spherion® Workplace Snapshot survey. 
“The results of this survey confirm what we know 
intuitively—that many workers find opportunities 
for romance where they work,” shared John Heins, 
senior vice president and chief human resources 
officer at Spherion. 

The workplace romance findings were collected 
using the Harris Interactive QuickQuery online om- 
nibus, an online survey fielded two to three times 
per week. A U.S. sample of 1,588 employed adults, 
aged 18 or older, were polled in a three-day period 
in January. Results were weighted to bring them in 
line with the actual U.S. population. 

According to the survey, nearly 40 percent 
of workers (30 percent of women, 47 percent of men) would 
consider dating a co-worker or have done so. Approximately 
25 percent (27 percent of men, 23 percent of women) of 
such romances result in marriage. While 41 percent of work- 
ers (47 percent of women, 36 percent of men) think an office 
romance will jeopardize their job security or advancement, 
42 percent conduct their romance openly. “The new wrinkle is 
the explosion of online venues such as blogs, YouTube, and 
social networking sites, which provide very public means for www.spherion.com; harrisinteractive.com; 
personal news to be shared,” commented Heins. “Becoming a www.boswellgroup.com 


target of gossip on the Internet does have the potential to affect 
career advancement and security.” 

Only 16 percent of workers’ employers have a policy regard- 
ing workplace romance. Although most of us will spend one-third 
of each day at work, Boswell Group’s founding psychoanalyst 
Kerry Sulkowicz reminds us that “if there’s any reporting relation- 
ship between the two [people involved], the less powerful person 
will be the one asked to change jobs or leave.” 


General to Specific 


The procedure of moving from general to more specific questions is sometimes called the funnel ap- 
proach. The objectives of this procedure are to learn the participant’s frame of reference and to extract 
the full range of desired information while limiting the distortion effect of earlier questions on later 
ones. This process may be illustrated with the following series of questions: 


1. How do you think this country is getting along in its relations with other countries? 
. How do you think we are doing in our relations with Iran? 

. Do you think we ought to be dealing with Iran differently than we are now? 

. (If yes) What should we be doing differently? 


. Some people say we should get tougher with Iran and others think we are too tough as it is; 
how do you feel about it?!® 


nan WwW WN 


The first question introduces the general subject and provides some insight into the participant’ s frame of refer- 
ence. The second question narrows the concern to a single country, while the third and fourth seek views on 
how the United States should deal with Iran. The fifth question illustrates a specific opinion area and would be 
asked only if this point of toughness had not been covered in earlier responses. Question 4 is an example of a 
branched question; the response to the previous question determines whether or not question 4 is asked of the 
participant. You might find it valuable to refer to Exhibit 7-6, “The Interview Question Hierarchy,” page 154. 
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>Exhibit 13-12 Question Sequencing 


Percent Answering Yes 


Question A. Asked First __B. Asked First 


A. Should the United States permit its citizens to join the French and British 45% 40% 
armies? 
B. Should the United States permit its citizens to join the German army? 31 22 


There is also a risk of interaction whenever two or more questions are related. Question-order in- 
fluence is especially problematic with self-administered questionnaires, because the participant is at 
liberty to refer back to questions previously answered. In an attempt to “correctly align” two responses, 
accurate opinions and attitudes may be sacrificed. Computer-administered and Web surveys have 
largely eliminated this problem. 

The two questions shown in Exhibit 13-12 were asked in a national survey at the start of World War II.'” 
Apparently, some participants who first endorsed enlistment with the Allies felt obliged to extend this 
privilege to joining the German army. When the decision was first made against joining the German army, 
a percentage of the participants felt constrained from approving the option to join the Allies. 


Question Groups and Transitions 


The last question-sequencing guideline suggests arranging questions to minimize shifting in subject 
matter and frame of reference. Participants often interpret questions in the light of earlier questions and 
miss shifts of perspective or subject unless they are clearly stated. Participants fail to listen carefully 
and frequently jump to conclusions about the import of a given question before it is completely stated. 
Their answers are strongly influenced by their frame of reference. Any change in subject by the inter- 
viewer may not register with them unless it is made strong and obvious. Most questionnaires that cover 
a range of topics are divided into sections with clearly defined transitions between sections to alert the 
participant to the change in frame of reference. Exhibit 13-11 provides a sample of a transition in a 
study when measurement questions changed to personal and family-related questions. 


Instructions 


Instructions to the interviewer or participant attempt to ensure that all participants are treated equally, 
thus avoiding building error into the results. Two principles form the foundation for good instructions: 
clarity and courtesy. Instruction language needs to be unfailingly simple and polite. 

Instruction topics include those for: 


° Terminating an unqualified participant—defining for the interviewer how to terminate an inter- 
view when the participant does not correctly answer the screen or filter questions. 


° Terminating a discontinued interview—defining for the interviewer how to conclude an inter- 
view when the participant decides to discontinue. 


° Moving between questions on an instrument—defining for an interviewer or participant how to 
move between questions or topic sections of an instrument (skip directions) when movement is 
dependent on the specific answer to a question or when branched questions are used. 


° Disposing of a completed questionnaire—defining for an interviewer or participant completing a 
self-administered instrument how to submit the completed questionnaire. 


In a self-administered questionnaire, instructions must be contained within the survey instrument. This 
may be as simple as “Click the submit button to send your answers to us.” Personal interviewer instruc- 
tions sometimes are in a document separate from the questionnaire (a document thoroughly discussed 
during interviewer training) or are distinctly and clearly marked (highlighted, printed in colored ink, or 
boxed on the computer screen or in a pop-up window) on the data collection instrument itself. Sample 
instructions are presented in Exhibit 13-13. 


Instrument Design for MindWriter 
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() Replacing an imprecise management question 

with specific measurement questions is an ex- 
lids hill ercise in analytical reasoning. We described that 
process incrementally in the MindWriter features in Chapters 3, 
4, and 13. In Chapter 3, Jason’s fact finding at MindWriter re- 
sulted in the ability to state the management dilemma in terms 
of management, research, and investigative questions. Adding 
context to the questions allowed Jason and Sara to construct 
the proposal described in Appendix A, Exhibit A-8. In Chapter 
12, they returned to the list of investigative questions and se- 
lected one question to use in testing their scaling approach. 
Here is a brief review of the steps Jason and Sara have taken so 
far and the measurement questions that have resulted. 


SYNOPSIS OF THE PROBLEM 
MindWriter Corporation’s new service and repair program for 
laptop computers, CompleteCare, was designed to provide a 
rapid response to customers’ service problems. Management 
has received several complaints, however. Management needs 
information on the program’s effectiveness and its impact on 
customer satisfaction. There is also a shortage of trained techni- 
cal operators in the company’s telephone center. The package 
courier is uneven in executing its pickup and delivery contract. 
Parts availability problems exist for some machine types. Occa- 
sionally, customers receive units that either are not fixed or are 
damaged in some way. 

Management question: What should be done to improve the 

CompleteCare program for MindWriter laptop repairs and 

servicing to enhance customer satisfaction? 


RESEARCH QUESTIONS 

1. Should the technical representatives be given more in- 
tensive training? 

2. Should ABC Courier Service be replaced by an overnight 
air transport? 

3. Should the repair diagnostic and repair sequencing op- 
erations be modified? 

4. Should the return packaging be modified to include pre- 
molded rigid foam inserts, conforming-expanding foam 
protection, or neither? 

5. Should metropolitan repair centers be established to 
complement or replace in-factory repair facilities? 


INVESTIGATIVE QUESTIONS 
a. Are customers’ expectations being met in terms of 
the time it takes to repair the systems? What is the 


customers’ overall satisfaction with the CompleteCare 
service program and the MindWriter product? 

b. How well is the call center helping customers? Is it help- 
ing them with instructions? What percentage of custom- 
ers’ technical problems is it solving without callbacks or 
subsequent repairs? How long must customers wait on 
the phone? 

c. How good is the transportation company? Does it pick 
up and deliver the system responsively? How long must 
customers wait for pickup and delivery? Are the laptops 
damaged due to package handling? 

d. How good is the repair group? What problems are most 
common? What repair processes are involved in fixing 
these problems? For what percentage of laptops is the 
repair completed within the promised time limit? Are cus- 
tomers’ problems fully resolved? Are there new problems 
with the newer models? How quickly are these problems 
diagnosed? 

e. How are repaired laptops packaged for return shipping? 
What is the cost of alternative rigid foam inserts and 
expandable-foam packaging? Would new equipment be 
required if the packaging were changed? Would certain 
shipping-related complaints be eliminated with new 
packaging materials? 


The extensive scope of the research questions and result- 
ing measurement questions forced MindWriter to reassess the 
scope of the desired initial research study, to determine where 
to concentrate its enhancement efforts. Management chose a 
descriptive rather than a prescriptive scope. 


MEASUREMENT QUESTIONS 

The measurement questions used for the self-administered 
online survey are shown in Exhibit 13-13.'° The first investiga- 
tive question in (a), above, is addressed in survey items 8, 5, 
and 8a, while the second question is addressed in items 6 and 
8a. Of the investigative questions in (b), the first two are ad- 
dressed as “responsiveness” and “technical competence” with 
telephone assistance in the questionnaire. The second two in- 
vestigative questions in (b) may be answered by accessing the 
company’s service database. The questionnaire’s three-part 
question on courier service parallels investigative question (c). 
Specific service deficiencies will be recorded in the “Comments/ 
Suggestions” section. Investigative questions under (d) and (e) 
are covered with questionnaire items 3, 4, and 5. Because ser- 
vice deficiencies reflected in item 5 may be attributed to both 
the repair facility and the courier, the reasons (items 1, 2, 3, 4, 
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>closeupcont’d 


and comments) will be cross-checked during analysis. Ques- use CompleteCare again. Finally, the last item will make known 
tionnaire item 6 uses the same language as the last investiga- the extent to which change is needed in CompleteCare by re- 
tive question in (a). Questionnaire item 7 is an extension of item vealing repurchase intention as linked to product and service 
6 but attempts to secure an impression of behavioral intent to experience. 


>Exhibit 13-13 Measurement Questions for the MindWriter Study 


MindWriter personal computers offer you ease of use and maintenance. When you need 
service, we want you to rely on CompleteCare, wherever you may be. That’s why we’re 
asking you to take a moment to tell us how well we’ve served you. 

Please answer the first set of questions using the following scale: MindWriter 
Met Few Met Some Met Most Met All Exceeded 
Expectations Expectations Expectations Expectations Expectations 
1 2 3 4 5 

1 2 3 4 5 


1. Telephone assistance with your problem: 
a. Responsiveness 
b. Technical competence 
2. The courier service's effectiveness: 
a. Arrangements 
b. Pickup speed 


c. Delivery speed 


OMOMOmO 


3. Speed of the overall repair process 
4. Resolution of the problem that prompted service/repair CE 


5. Condition of your MindWriter on arrival O 


PPORRORIOMIORIO 
P2ORRORIOMOMO 
PPORFOMIOMIOMO 
PPORVOMIOMIOMO 


6. Overall impression of CompleteCare’s effectiveness O 


How likely would you be to... 
Very Somewhat Neither Somewhat Very 
Unlikely Unlikely —_ Unlikely Likely Likely 


nor Likely 
7. Use CompleteCare on another occasion O O O O O 
8. Repurchase another MindWriter based on: 
a. Service/repair experience O O O O O 
b. Product Performance O O O O O 


Please share any additional comments or suggestions 


How may we contact you to follow up on any problems you have experienced? 


Last Name First Name Email 
City State Zipcode Phone 
Service Code 
Thank you for your participation. 
_ SUBMIT _ 
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Conclusion 


The role of the conclusion is to leave the participant with the impression that his or her involvement has 
been valuable. Subsequent researchers may need this individual to participate in new studies. If every 
interviewer or instrument expresses appreciation for participation, cooperation in subsequent studies is 
more likely. A sample conclusion is shown in Exhibit 13-13. 


Overcoming Instrument Problems 


There is no substitute for a thorough understanding of question wording, question content, and ques- 
tion sequencing issues. However, the researcher can do several things to help improve survey results, 
among them: 


¢ Build rapport with the participant. 

¢ Redesign the questioning process. 

e Explore alternative response strategies. 

¢ Use methods other than surveying to secure the data. 


¢ Pretest all the survey elements. (See Appendix 13b available from the text Online Learning Center.) 


Build Rapport with the Participant 


Most information can be secured by direct undisguised questioning if rapport has been developed. Rap- 
port is particularly useful in building participant interest in the project, and the more interest partici- 
pants have, the more cooperation they will give. One can also overcome participant unwillingness by 
providing some material compensation for cooperation. This approach has been especially successful 
in mail surveys and is increasingly used in Web surveys. 

The assurance of confidentiality also can increase participants’ motivation. One approach is to give 
discrete assurances, both by question wording and by interviewer comments and actions, that all types 
of behavior, attitudes, and positions on controversial or sensitive subjects are acceptable and normal. 
Where you can say so truthfully, guarantee that participants’ answers will be used only in combined 
statistical totals (aggregate data), not matched to an individual participant. If participants are convinced 
that their replies contribute to some important purpose, they are more likely to be candid, even about 
taboo topics. If a researcher’s organization uses an Institutional Review Board to review surveys before 
use, the board may require an instruction indicating that any response—in fact, participation—is volun- 
tary. This is especially important where surveys are used with internal publics (employees). 


Redesign the Questioning Process 


You can redesign the questioning process to improve the quality of answers by modifying the admin- 
istrative process and the response strategy. Most online surveys, while not actually anonymous as each 
respondent is connected to an IP address, a cell number, or an email, leave the perception of being more 
anonymous. For paper surveys, we might show that confidentiality is indispensable to the administra- 
tion of the survey by using a group administration of questionnaires, accompanied by a ballot-box 
collection procedure. Even in face-to-face interviews, the participant may fill in the part of the ques- 
tionnaire containing sensitive information and then seal the entire instrument in an envelope. Although 
this does not guarantee confidentiality, it does suggest it. Additionally, for online surveys, we might 
have respondents enter a provided code number for identity purposes, rather than personal information. 

We can also develop appropriate questioning sequences that will gradually lead a participant from 
“safe” topics to those that are more sensitive. As already noted in our discussion of disguised questions, 
indirect questioning (using projective techniques) is a widely used approach for securing opinions on 
sensitive topics. The participants are asked how “other people” or “people you know” feel about a 
topic. It is assumed the participants will reply in terms of their own attitudes and experiences, but this 
outcome is hardly certain. Indirect questioning may give a good measure of the majority opinion on a 
topic but fail to reflect the views either of the participant or of minority segments. 
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With certain topics, it is possible to secure answers by using a proxy code. When we seek family in- 
come groups, in an intercept survey we can hand the participant a card with income brackets like these: 


A. Under $25,000 per year. 

B. $25,000 to $49,999 per year. 
C. $50,000 to $74,999 per year. 
D. $75,000 and over per year. 


The participant is then asked to report the appropriate bracket as either A, B, C, or D. For some reason, par- 
ticipants are more willing to provide such an obvious proxy measure than to verbalize actual dollar values. 


Explore Alternative Response Strategies 


When drafting the original question, try developing positive, negative, and neutral versions of each 
type of question. This practice dramatizes the problems of bias, helping you to select question word- 
ing that minimizes such problems. Sometimes use an extreme version of a question rather than the 
expected one. 

Minimize nonresponses to particular questions by recognizing the sensitivity of certain topics. In 
a self-administered instrument, for example, asking a multiple-choice question about income or age, 
where incomes and ages are offered in ranges, is usually more successful than using a free-response 
question (such as “What is your age, please? ee 


The Value of Pretesting 


The final step toward improving survey results is pretesting, the assessment of questions and instru- 
ments before the start of a study (see Exhibits 13-1, 13-2, and 13-10). There are abundant reasons 
for pretesting individual questions, questionnaires, and interview schedules: (1) discovering ways to 
increase participant interest, (2) increasing the likelihood that participants will remain engaged to the 
completion of the survey, (3) discovering question content, wording, and sequencing problems, (4) dis- 
covering target question groups where researcher training is needed, and (5) exploring ways to improve 
the overall quality of survey data. 


Most of what we know about pretesting is prescriptive. According to contemporary authors, there are no general 
principles of good pretesting, no systematization of practice, no consensus about expectations, and we rarely leave 
records for each other. How a pretest was conducted, what investigators learned from it, how they redesigned their 
questionnaire on the basis of it—these matters are reported only sketchily in research reports, if at all.'° 


Nevertheless, pretesting not only is an established practice for discovering errors but also is useful 
for training the research team. Ironically, professionals who have participated in scores of stud- 
ies are more likely to pretest an instrument than is a beginning researcher hurrying to complete a 
project. Revising questions five or more times is not unusual. Yet inexperienced researchers often 
underestimate the need to follow the design-test-revise process. We devote Appendix 13b to pre- 
testing; it’s available from the text’s Online Learning Center. 


1. The instrument design process starts with a comprehensive 2 Several choices must be made in designing a communica- 


list of investigative questions drawn from the management- 
research question hierarchy. Instrument design is a three- 
phase process with numerous issues within each phase: 

(a) developing the instrument design strategy, (b) construct- 
ing and refining the measurement questions, and (c) drafting 
and refining the instrument. 


tion study instrument. Surveying can be a face-to-face inter- 
view, or it can be much less personal, using indirect media 
and self-administered questionnaires. The questioning 
process can be unstructured, as in an IDI, or the questions 
can be clearly structured. Responses may be unstructured 
and open-ended or structured with the participant choosing 


from a list of possibilities. The degree to which the objec- 
tives and intent of the questions should be disguised must 
also be decided. 


3 Instruments obtain three general classes of information. 
Target questions address the investigative questions and 
are the most important. Classification questions concern 
participant characteristics and allow participants’ answers 
to be grouped for analysis. Administrative questions iden- 
tify the participant, interviewer, and interview location and 
conditions. 


Question construction involves three critical decision areas. 
They are (a) question content, (6) question wording, and 

(c) response strategy. Question content should pass the fol- 
lowing tests: Should the question be asked? Is it of proper 
scope? Can and will the participant answer adequately? 

Question wording difficulties exceed most other sources 
of distortion in surveys. Retention of a question should be 
confirmed by answering these questions: Is the question 
stated in terms of a shared vocabulary? Does the vocabu- 
lary have a single meaning? Does the question contain mis- 
leading assumptions? Is the wording biased? Is it correctly 
personalized? Are adequate alternatives presented? 

The study’s objective and participant factors affect the 
decision of whether to use open-ended or closed ques- 
tions. Each response strategy generates a specific level of 
data, with available statistical procedures for each scale 
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type influencing the desired response strategy. Participant 
factors include level of information about the topic, degree 
to which the topic has been thought through, ease of 
communication, and motivation to share information. The 
decision is also affected by the interviewer's perception of 
participant factors. 

Both dichotomous response and multiple-choice ques- 
tions are valuable, but on balance the latter are preferred if 
only because few questions have only two possible answers. 
Checklist, rating, and ranking strategies are also common. 


Question sequence can drastically affect participant willing- 
ness to cooperate and the quality of responses. Generally, 
the sequence should begin with efforts to awaken the par- 
ticipant’s interest in continuing the interview. Early questions 
should be simple rather than complex, easy rather than 
difficult, nonthreatening, and obviously germane to the an- 
nounced objective of the study. Frame-of-reference changes 
should be minimal, and questions should be sequenced so 
that early questions do not distort replies to later ones. 


Sources of questions for the construction of questionnaires 
include the literature on related research and sourcebooks 
of scales and questionnaires. Borrowing items has atten- 
dant risks, such as time and situation-specific problems or 
reliability and validity. Incompatibility of language and idiom 
also needs to be considered. 


oe 


administrative questions 302 dummy table 300 rating question 311 


branched question 317 free-response question 308 recency effect 310 


buffer question 318 interview schedule 302 screen questions (filter questions) 316 


checklist 311 leading questions 304 structured response 306 


classification questions 302 multiple-choice question 308 target questions 302 


dichotomous question 308 pretesting 324 structured, 302 


disguised question 299 primacy effect 310 unstructured, 302 


double-barreled question 309 ranking question 312 unstructured response 306 


>di ion lOn 
Terms in Review 2 Why is the survey technique so popular? When is it not 
1. Distinguish between: appropriate? 
a Direct and indirect questions. 3 What special problems do open-ended questions have? 


How can these be minimized? In what situations are open- 
ended questions most useful? 


b Open-ended and closed questions. 
c Research, investigative, and measurement questions. 
4 Why might a researcher wish to disguise the objective of a 


d Alternative response strategies. 
P 9 study? 
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5 One of the major reasons why survey research may not be 
effective is that the survey instruments are less useful than 
they should be. What would you say are the four possible 
major faults of survey instrument design? 


6 Why is it desirable to pretest survey instruments? What 
information can you secure from such a pretest? How 
can you find the best wording for a question on a 
questionnaire? 

7 One design problem in the development of survey instru- 
ments concerns the sequence of questions. What sug- 
gestions would you give to researchers designing their first 
questionnaire? 

8 One of the major problems facing the designer of a survey 
instrument concerns the assumptions made. What are the 
major “problem assumptions”? 

Making Research Decisions 

9 Following are six questions that might be found on ques- 
tionnaires. Comment on each as to whether or not it is a 
good question. If it is not, explain why. (Assume that no 
lead-in or screening questions are required. Judge each 
question on its own merits.) 

a Do you read National Geographic magazine regularly? 


b What percentage of your time is spent asking for infor- 
mation from others in your organization? 


When did you first start chewing gum? 


d How much discretionary buying power do you have 
each year? 


e Why did you decide to attend Big State University? 
f Do you think the president is doing a good job now? 


10 Inaclass project, students developed a brief self- 
administered questionnaire by which they might quickly 
evaluate a professor. One student submitted the following 
instrument. Evaluate the questions asked and the format 
of the instrument. 


Professor Evaluation Form 

1. Overall, how would you rate this professor? 
Good C1 Fair Poor 
2. Does this professor 


a. Have good class delivery? 

. Know the subject? 

. Have a positive attitude toward the subject? 
. Grade fairly? 


. Have a sense of humor? 


— OO iO oO oF 


Use audiovisuals, case examples, or other classroom 
aids? 


g. Return exams promptly? 
3. What is the professor’s strongest point? 


4. What is the professor’s weakest point? 


5. What kind of class does the professor teach? 
6. Is this course required? 


7. Would you take another course from this professor? 


11. Assume the American Society of Training Directors is 
studying its membership in order to enhance member ben- 
efits and attract new members. Below is a copy of a cover 
letter and mail questionnaire received by a member of the 
society. Please evaluate the usefulness and tone of the 
letter and the questions and format of the instrument. 


Dear ASTD Member: 


The ASTD is evaluating the perception of value of membership 
among its members. Enclosed is a short questionnaire and a 
return envelope. | hope you will take a few minutes and fill out 
the questionnaire as soon as possible, because the sooner the 
information is returned to me, the better. 

Sincerely, 

Director of Membership 

Questionnaire 

Directions: Please answer as briefly as possible. 


1. With what company did you enter the field of training? 


2. How long have you been in the field of training? 


3. How long have you been in the training department of the 
company with which you are presently employed? 


4. How long has the training department in your company 
been in existence? 


5. Is the training department a subset of another department? 
If so, what department? 


6. For what functions (other than training) is your department 
responsible? 


7. How many people, including yourself, are in the training de- 
partment of your company (local plant or establishment)? 


8. What degrees do you hold and from what institutions? 


Major Minor 


9. Why were you chosen for training? What special qualifica- 
tions prompted your entry into training? 


10. What experience would you consider necessary for an indi- 
vidual to enter into the field of training with your company? 
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Include both educational requirements and actual From Concept to Practice 

experience. 14 Using Exhibits 13-1, 18-4, and 13-10, develop the flow- 
chart for the Albany Outpatient Laser Clinic study in the 
opening vignette. 


From the Headlines 
15 Government economic data reveals that young adults, 


Bringing en to ane not middle-aged or older adults, are having the most dif- 
12 Design the introduction of the Albany Outpatient Laser ficult time in today’s economy. Although the nation’s labor 
Clinic survey, assuming i will continue to be a self- market shows a decline in the unemployment rate, the 
administered questionnaire. percentage of young adults, ages 18 to 24, currently em- 
13 To evaluate whether presurgery patient attitudes affect ployed (54%) is at the lowest level since government data 
recovery and ultimate patient satisfaction with the Albany collection began in 1948. If you were working for a national 
Outpatient Laser Clinic, design a question for the self- survey organization doing a general public survey of young 
administered survey. (You may wish to review the opening adults and older adults, what topics and questions would 
vignettes in this chapter and Chapter 9.) you design into your survey to elaborate on this finding? 
* 
re 
ee Akron Children’s Hospital NCRCC: Teeing Up and New Strategic Direction 
Calling Up Attendance ee Ohio Lottery: Innovative Research Design Drives 
Winning 


Campbell-Ewald: R-E-S-P-E-C-T 


Spells Loyalty ee Pebble Beach Co. 
Can Research Rescue the Red Cross? Proofpoint: Capitalizing on a Reporter’s Love of 
Statistics 


Inquiring Minds Want to 


Know—NOW! & Starbucks, Bank One, and Visa Launch Starbucks 
Duetto Visa 


Marcus Thomas LLC Tests 
Hypothesis for Troy-Bilt Creative 
Development 


USTA: Come Out Swinging 


Volkswagen’s Beetle 


Mastering Teacher Leadership 


“You will find a description of each case in the Case Index section of the textbook. Check the Case index to determine whether a 
case provides data, the research instrument, video, or other supplementary material. Written cases are downloadable from the text 
website (www.mhhe.com/cooper1 2e). All video material and video cases are available from the Online Learning Center. The film reel 
icon indicates a video case or video material relevant to the case. 


>online learnin nter ndi 


You'll find the following appendix available from the Online Learning Center (www.mhhe.com/cooper1 2e) to supplement the 
content of this chapter: 
Appendix 13b: Pretesting Options and Discoveries 


Crafting Effective Measurement Questions 


Numerous issues influence whether the questions we ask 
on questionnaires generate the decision-making data that 
managers sorely need. Each of the issues summarized in 
Exhibit 13-5 is developed more fully here. 


Question Content 
Should This Question Be Asked? 


Purposeful versus Interesting Questions that 
merely produce “interesting information” cannot be justi- 
fied on either economic or research grounds. Challenge each 
question’s function. Does it contribute significant informa- 
tion toward answering the research question? Will its omis- 
sion limit or prevent the thorough analysis of other data? 
Can we infer the answer from another question? A good 
question designer knows the value of learning more from 
fewer questions. 


Is the Question of Proper Scope 
and Coverage? 


Incomplete or Unfocused We can test this con- 
tent issue by asking, Will this question reveal all we 
need to know? We sometimes ask participants to reveal 
their motivations for particular behaviors or attitudes by 
asking them, Why? This simple question is inadequate 
to probe the range of most causal relationships. When 
studying product use behavior, for example, we learn 
more by directing two or three questions on product use 
to the heavy-use consumer and only one question to the 
light user. 

Questions are also inadequate if they do not provide 
the information you need to interpret responses fully. If 
you ask about the Albany Clinic’s image for quality pa- 
tient care, do different groups of patients or those there 
for the first versus the third time have different attitudes? 
To evaluate relative attitudes, do you need to ask the same 
question about other companies? In the original Albany 
Clinic survey, participants were asked, “Have you ever 
had or been treated for a recent cold or flu?” If participants 
answer yes, what exactly have they told the researcher that 
would be of use to the eye surgeon? Wouldn’t it be likely 
that the surgeon is interested in medication taken to treat 
colds or flu within, say, the prior 10 days? This question 


also points to two other problems of scope and coverage: 
the double-barreled question and the imprecise question. 


Double-Barreled Questions Does the question 
request so much content that it should be broken into two 
or more questions? While reducing the overall number of 
questions in a study is highly desirable, don’t try to ask 
double-barreled questions. The Albany Clinic question 
about flu (“Have you ever had or been treated for a re- 
cent cold or flu?’’) fires more than two barrels. It asks four 
questions in all (Ever had cold? Ever had flu? Been treated 
for cold? Been treated for flu?). 

Here’s another common example posed to menswear 
retailers: “Are this year’s shoe sales and gross profits 
higher than last year’s?” Couldn’t sales be higher with 
stagnant profits, or profits higher with level or lower 
sales? This second example is more typical of the problem 
of double-barreled questions. 

A less obvious double-barreled question is the question 
we ask to identify a family’s or a group’s TV station pref- 
erence. Since a single station is unlikely, a better question 
would ask the station preference of each family member 
separately or, alternatively, screen for the group member 
who most often controls channel selection on Monday 
evenings during prime time. Also, it’s highly probable that 
no one station would serve as an individual’s preferred sta- 
tion when we cover a wide range of time (8 to 11 p.m.). 
This reveals another problem, the imprecise question. 


Precision To test a question for precision, ask, Does 
the question ask precisely what we want and need to know? 
We sometimes ask for a participant’s income when we re- 
ally want to know the family’s total annual income before 
taxes in the past calendar year. We ask what a participant 
purchased “last week” when we really want to know what 
he or she purchased in a “typical 7-day period during the 
past 90 days.” The Albany Clinic’s patients were asked for 
cold and flu history during the time frame “ever.” It is hard 
to imagine an adult who has never experienced a cold or 
flu and equally hard to assume an adult hasn’t been treated 
for one or both at some time in his or her life. 

A second precision issue deals with common vocabu- 
lary between researcher and participant. To test your ques- 
tion for this problem, ask, Do I need to offer operational 
definitions of concepts and constructs used in the question? 
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Can the Participant Answer Adequately? 


Time for Thought Although the question may ad- 
dress the topic, is it asked in such a way that the participant 
will be able to frame an answer, or is it reasonable to as- 
sume that the participant can determine the answer? This 
is also a question that drives sample design, but once the 
ideal sample unit is determined, researchers often assume 
that participants who fit the sample profile have all the an- 
swers, preferably on the tips of their tongues. To frame a 
response to some questions takes time and thought; such 
questions are best left to self-administered questionnaires. 


Participation at the Expense of Accuracy _Par- 
ticipants typically want to cooperate in interviews; thus 
they assume giving any answer is more helpful than de- 
nying knowledge of a topic. Their desire to impress the 
interviewer may encourage them to give answers based 
on no information. A classic illustration of this problem 
occurred with the following question:' “Which of the fol- 
lowing statements most closely coincides with your opin- 
ion of the Metallic Metals Act?” The response pattern 
shows that 70 percent of those interviewed had a fairly 
clear opinion of the Metallic Metals Act; however, there 
is no such act. The participants apparently assumed that 
if a question was asked, they should provide an answer. 
Given reasonable-sounding choices, they selected one 
even though they knew nothing about the topic. 

To counteract this tendency to respond at any cost, 
filter or screen questions are used to qualify a participant’s 
knowledge. If the MindWriter service questionnaire is dis- 
tributed via mail to all recent purchasers of MindWriter 
products, we might ask, “Have you required service for 
your laptop since its purchase?” Only those for whom ser- 
vice was provided could supply the detail and scope of 
the responses indicated in the investigative question list. 
If such a question is asked in a phone interview, we would 
call the question a screen, because it is being used to de- 
termine whether the person on the other end of the phone 
line is a qualified sample unit. This same question asked 
on a computer-administered questionnaire would likely 
branch or skip the participant to a series of classification 
questions. 

Assuming that participants have prior knowledge or 
understanding may be risky. The risk is getting many an- 
swers that have little basis in fact. The Metallic Metals Act 
illustration may be challenged as unusual, but in another 
case a Gallup report revealed that 45 percent of the persons 
surveyed did not know what a “lobbyist in Washington” 
was and 88 percent could not give a correct description of 
“Jurisdictional strike.”* This points to the need for opera- 
tional definitions as part of question wording. 


Presumed Knowledge The question designer 
should consider the participants’ information level when 
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determining the content and appropriateness of a question. 
In some studies, the degree of participant expertise can be 
substantial, and simplified explanations are inappropriate 
and discourage participation. In asking the public about 
gross margins in menswear stores, we would want to be 
sure the “general-public” participant understands the na- 
ture of “gross margin.” If our sample unit were a merchant, 
explanations might not be needed. A high level of knowl- 
edge among our sample units, however, may not eliminate 
the need for operational definitions. Among merchants, 
gross margin per unit in dollars is commonly accepted as 
the difference between cost and selling price; but when 
offered as a percentage rather than a dollar figure, it can 
be calculated as a percentage of unit selling price or as a 
percentage of unit cost. A participant answering from the 
“cost” frame of reference would calculate gross margin at 
100 percent; another participant, using the same dollars 
and the “‘selling price” frame of reference, would calculate 
gross margin at 50 percent. If a construct is involved and 
differing interpretations of a concept are feasible, opera- 
tional definitions may still be needed. 


Recall and Memory Decay The adequacy prob- 
lem also occurs when you ask questions that overtax par- 
ticipants’ recall ability. People cannot recall much that 
has happened in their past, unless it was dramatic. Your 
mother may remember everything about your arrival if 
you were her first child: the weather, time of day, even 
what she ate prior to your birth. If you have several sib- 
lings, her memory of subsequent births may be less com- 
plete. If the events surveyed are of incidental interest to 
participants, they will probably be unable to recall them 
correctly even a short time later. An unaided recall ques- 
tion, “What radio programs did you listen to last night?” 
might identify as few as 10 percent of those individuals 
who actually listened to a program.? 


Balance (General versus Specific) Answering 
adequacy also depends on the proper balance between 
generality and specificity. We often ask questions in terms 
too general and detached from participants’ experiences. 
Asking for average annual consumption of a product may 
make an unrealistic demand for generalization on people 
who do not think in such terms. Why not ask how often 
the product was used last week or last month? Too often 
participants are asked to recall individual use experiences 
over an extended time and to average them for us. This 
is asking participants to do the researcher’s work and en- 
courages substantial response errors. It may also contrib- 
ute to a higher refusal rate and higher discontinuation rate. 

There is a danger in being too narrow in the time frame 
applied to behavior questions. We may ask about movie at- 
tendance for the last seven days, although this is too short 
a time span on which to base attendance estimates. It may 
be better to ask about attendance, say, for the last 30 days. 


330 >part Ill The Sources and Collection of Data 


>Exhibit 13a-1 A Test of Alternative Response Strategies 


A. What is your favorite brand of ice cream? 


B. Some people have a favorite brand of ice cream, while others do not have a favorite brand. In which group are you? (please check) 


(41 do not have a favorite brand of ice cream. 
(41 have a favorite brand of ice cream. 
What is your favorite (if you have a favorite)? 


>Exhibit 13a-2 Results of Alternative Response Strategies Test 


Response 


Named a favorite brand 

Named a favorite flavor rather than a brand 
Had no favorite brand 

Total 


*Significant difference at the 0.001 level. 


There are no firm rules about this generality-specificity 
problem. Developing the right level of generality depends 
on the subject, industry, setting, and experience of the 
question designer. 


Objectivity The ability of participants to answer ad- 
equately is also often distorted by questions whose content 
is biased by what is included or omitted. The question may 
explicitly mention only the positive or negative aspects of 
the topic or make unwarranted assumptions about the par- 
ticipant’s position. Consider Exhibit 13a-1, an experiment 
in which two forms of a question were asked. Fifty-seven 
randomly chosen graduate business students answered 
version A, and 56 answered version B. Their responses 
are shown in Exhibit 13a-2. The probable cause of the dif- 
ference in level of brand preference expressed is that A is 
an unsupported assumption. It assumes and suggests that 
everyone has a favorite brand of ice cream and will re- 
port it. Version B indicates the participant need not have 
a favorite. 

A deficiency in both versions is that about one partici- 
pant in five misinterpreted the meaning of the term brand. 
This misinterpretation cannot be attributed to low educa- 
tion, low intelligence, lack of exposure to the topic, or 
quick or lazy reading of the question. The subjects were 
students who had taken at least one course in marketing in 
which branding was prominently treated.* 


Will the Participants Answer Willingly? 


Sensitive Information Even if participants have 
the information, they may be unwilling to give it. Some 


*Word confusion difficulties are discussed later in this appendix. 


Version A Version B 
aon 389%* 
19 18 
_4 _43 
100% 100% 
n=57 n=56 


topics are considered too sensitive to discuss with strang- 
ers. These vary from person to person, but one study sug- 
gests the most sensitive topics concern money matters and 
family life.* More than one-fourth of those interviewed 
mentioned these as the topics about which they would be 
“least willing to answer questions.” Participants of lower 
socioeconomic status also included political matters in 
this “least willing” list. 

Participants also may be unwilling to give correct an- 
swers for ego reasons. Many exaggerate their incomes, the 
number of cars they own, their social status, and the amount 
of high-prestige literature they read. They also minimize 
their age and the amount of low-prestige literature they 
read. Many participants are reluctant to try to give an ad- 
equate response. Often this will occur when they see the 
topic as irrelevant to their own interests or to their percep- 
tion of the survey’s purpose. They participate halfheartedly, 
often answer with “don’t know,” give negative replies, give 
stereotypical responses, or refuse to be interviewed. 

You can learn more about crafting questions dealing 
with sensitive information by reading “Measuring Atti- 
tudes on Sensitive Subjects” on the text website. 


Question Wording 


Shared Vocabulary Because surveying is an ex- 
change of ideas between interviewer and participant, each 
must understand what the other says, and this is possible 
only if the vocabulary used is common to both parties.° 
Two problems arise. First, the words must be simple 
enough to allow adequate communication with persons of 
limited education. This is dealt with by reducing the level 
of word difficulty to simple English words and phrases 
(more is said about this in the section on word clarity). 


Technical language is the second issue. Even highly 
educated participants cannot answer questions stated in 
unfamiliar technical terms. Technical language also poses 
difficulties for interviewers. In one study of how corpo- 
ration executives handled various financial problems, in- 
terviewers had to be conversant with technical financial 
terms. This necessity presented the researcher with two 
alternatives—hiring people knowledgeable in finance and 
teaching them interviewing skills or teaching financial 
concepts to experienced interviewers.° This vocabulary 
problem also exists in situations where similar or identical 
studies are conducted in different countries and multiple 
languages. 

A great obstacle to effective question wording is the 
choice of words. Questions to be asked of the public 
should be restricted to the 2,000 most common words 
in the English language.’ Even the use of simple words 
is not enough. Many words have vague references or 
meanings that must be gleaned from their context. In a 
repair study, technicians were asked, “How many radio 
sets did you repair last month?” This question may 
seem unambiguous, but participants interpreted it in two 
ways. Some viewed it as a question of them alone; oth- 
ers interpreted “you” more inclusively, as referring to 
the total output of the shop. There is also the possibility 
of misinterpreting “last month,” depending on the tim- 
ing of the questioning. Using “during the last 30 days” 
would be much more precise and unambiguous. Typi- 
cal of the many problem words are these: any, could, 
would, should, fair, near, often, average, and regular. 
One author recommends that after stating a question as 
precisely as possible, we should test each word against 
this checklist: 


¢ Does the word chosen mean what we intend? 


¢ Does the word have multiple meanings? If so, does 
the context make the intended meaning clear? 


¢ Does the word chosen have more than one pronunci- 
ation? Is there any word with similar pronunciation 
with which the chosen word might be confused? 


e Is a simpler word or phrase suggested or possible?® 


We cause other problems when we use abstract con- 
cepts that have many overtones or emotional qualifica- 
tions.” Without concrete referents, meanings are too vague 
for the researcher’s needs. Examples of such words are 
business, government, and society. 

Shared vocabulary issues are addressed by using the 


following: 


e Simple rather than complex words. 
¢ Commonly known, unambiguous words. 
e Precise words. 


e Interviewers with content knowledge. 


>chapter 13 Questionnaires and Instruments 331 


Unsupported Assumptions Unwarranted assump- 
tions contribute to many problems of question wording. 
A metropolitan newspaper, Midwest Daily, conducted a 
study in an attempt to discover what readers would like 
in its redesigned lifestyle section. One notable question 
asked readers: “Who selects your clothes? You or the 
man in your life?” In this age of educated, working, in- 
dependent women, the question managed to offend a 
significant portion of the female readership. In addition, 
Midwest Daily discovered that many of its female readers 
were younger than researchers originally assumed and the 
only man in their lives was their father, not the spousal 
or romantic relationship alluded to by the questions that 
followed. Once men reached this question, they assumed 
that the paper was interested in serving only the needs of 
female readers. The unwarranted assumptions built into 
the questionnaire caused a significantly smaller response 
rate than expected and caused several of the answers to be 
uninterpretable. 


Frame of Reference Inherent in word meaning 
problems is also the matter of a frame of reference. 
Each of us understands concepts, words, and expres- 
sions in light of our own experience. The U.S. Bureau 
of the Census wanted to know how many people were 
in the labor market. To learn whether a person was em- 
ployed, it asked, “Did you do any work for pay or profit 
last week?” The researchers erroneously assumed there 
would be a common frame of reference between the in- 
terviewer and participants on the meaning of work. Un- 
fortunately, many persons viewed themselves primarily 
or foremost as homemakers or students. They failed to 
report that they also worked at a job during the week. 
This difference in frame of reference resulted in a consis- 
tent underestimation of the number of people working in 
the United States. 

In a subsequent version of the study, this question 
was replaced by two questions, the first of which sought 
a statement on the participant’s major activity during the 
week. If the participant gave a nonwork classification, a 
second question was asked to determine if he or she had 
done any work for pay besides this major activity. This re- 
vision increased the estimate of total employment by more 
than | million people, half of them working 35 hours or 
more per week.!° 

The frame of reference can be controlled in two ways. 
First, the interviewer may seek to learn the frame of refer- 
ence used by the participant. When asking participants to 
evaluate their reasons for judging a retail store as unat- 
tractive, the interviewer must learn the frames of refer- 
ence they use. Is the store being evaluated in terms of its 
particular features and layout, the failure of management 
to respond to a complaint made by the participant, the 
preference of the participant for another store, or the par- 
ticipant’s recent difficulty in returning an unwanted item? 
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>Exhibit 13a-3 Split Test of Alternative Question Wording 


Should the United States do any of the following at this time? 


A. Increase our armed forces further, even if it means more taxes. 


Eighty-eight percent of those answering question A thought the armed forces should be increased, while only 79 percent of those 


answering question B favored increasing the armed forces. 


Source: Hadley Cantril, ed., Gauging Public Opinion (Princeton, NJ: Princeton University Press, 1944), p. 48. 


Second, it is useful to specify the frame of reference 
for the participant. In asking for an opinion about the new 
store design, the interviewer might specify that the ques- 
tion should be answered based on the participant’s opinion 
of the layout, the clarity and placement of signage, the ease 
of finding merchandise, or another frame of reference. 


Biased Wording _ Bias is the distortion of responses 
in one direction. It can result from many of the problems 
already discussed, but word choice is often the major 
source. Obviously such words or phrases as politically 
correct or fundamentalist must be used with great care. 
Strong adjectives can be particularly distorting. One al- 
leged opinion survey concerned with the subject of prepa- 
ration for death included the following question: “Do you 
think that decent, low-cost funerals are sensible?” Who 
could be against anything that is decent or sensible? There 
is a question about whether this was a legitimate survey or 
a burial service sales campaign, but it shows how sugges- 
tive an adjective can be. 

Congressional representatives have been known to use 
surveys as a means of communicating with their constitu- 
encies. Questions are worded, however, to imply the issue 
stance that the representative favors. Can you tell the rep- 
resentative’s stance in the following question? 


Example: Would you have me vote for a balanced 
budget if it means higher costs for the sup- 
plemental Social Security benefits that you 
have already earned? 


We can also strongly bias the participant by using 
prestigious names in a question. In a historic survey on 
whether the war and navy departments should be com- 
bined into a single defense department, one survey said, 
“General Eisenhower says the army and navy should be 
combined,” while the other version omitted his name. 
Given the first version (name included), 49 percent of the 
participants approved of having one department; given the 
second version, only 29 percent favored one department."! 
Just imagine using Kobe Bryant’s or Dirk Nowitzki’s 


name in a survey question asked of teen boys interested in 
basketball. The power of aspirational reference groups to 
sway opinion and attitude is well established in advertis- 
ing; it shouldn’t be underestimated in survey design. 

We also can bias response through the use of superla- 
tives, slang expressions, and fad words. These are best 
excluded unless they are critical to the objective of the ques- 
tion. Ethnic references should also be stated with care. 


Personalization How personalized should a ques- 
tion be? Should we ask, “What would you do about...?” 
Or should we ask, “What would people with whom you 
work do about... ?” The effect of personalization is shown 
in a classic example reported by Cantril.'* A split test-—in 
which a portion of the sample received one question, with 
another portion receiving a second question—was made 
of a question concerning attitudes about the expansion of 
U.S. armed forces in 1940, as noted in Exhibit 13a-3. 

These and other examples show that personalizing 
questions changes responses, but the direction of the influ- 
ence is not clear. We cannot tell whether personalization 
or no personalization is superior. Perhaps the best that can 
be said is that when either form is acceptable, we should 
choose that which appears to present the issues more real- 
istically. If there are doubts, then split survey versions 
should be used (one segment of the sample should get one 
question version, while a second segment should receive 
the alternative question version). 


Adequate Alternatives Have we adequately ex- 
pressed the alternatives with respect to the purpose of the 
question? It is usually wise to express each alternative ex- 
plicitly to avoid bias. This is illustrated well with a pair 
of questions that were asked of matched samples of par- 
ticipants.'? The question forms that were used are noted in 
Exhibit 13a-4. 

Often the above issues are simultaneously present in 
a single question. Exhibit 13a-5 reveals several questions 
drawn from actual mail surveys. We’ ve identified the prob- 
lem issues and suggest one solution for improvement. 


>Exhibit 13a-4 Expressing Alternatives 
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The way a question is asked can influence the results. Consider these two alternative questions judging companies’ images in the 


community in the face of layoffs: 


A. Do you think most manufacturing companies that lay off workers during slack periods could arrange things to avoid layoffs 


and give steady work right through the year? 


B. Do you think most manufacturing companies that lay off workers in slack periods could avoid layoffs and provide steady work 


right through the year, or do you think layoffs are unavoidable? 


The Results: 


Source: Hadley Cantril, ed., Gauging Public Opinion (Princeton, NJ: Princeton University Press, 1944), p. 48. 


While the suggested improvement might not be the only 
possible solution, it does correct the issues identified. 
What other solutions could be applied to correct the prob- 
lems identified? 


Response Strategy 


The objectives of the study; characteristics of participants, 
especially their level of information, level of motivation 
to participate, and ease of communication; the nature of 
the topic(s) being studied; the type of scale needed; and 
your analysis plan dictate the response strategy. Examples 
of the strategies described in Chapter 13 and discussed in 
detail in Chapters 11 and 12 are found in Exhibit 13-6. 


Objective of the Study [If the objective of the 
question is only to classify the participant on some stated 
point of view, then the closed question will serve well. 
Assume you are interested only in whether a participant 
approves or disapproves of a certain corporate policy. A 
closed question will provide this answer. This response 
strategy ignores the full scope of the participant’s opinion 
and the events that helped shape the attitude at its foun- 
dation. If the objective is to explore this wider territory, 
then an open-ended question (free-response strategy) is 
preferable. 

Open-ended questions are appropriate when the ob- 
jective is to discover opinions and degrees of knowledge. 
They are also appropriate when the interviewer seeks 
sources of information, dates of events, and suggestions 
or when probes are used to secure more information. 
When the topic of a question is outside the partici- 
pant’s experience, the open-ended question may offer 
the better way to learn his or her level of information. 


Closed questions are better when there is a clear frame 
of reference, the participant’s level of information is 
predictable, and the researcher believes the participant 
understands the topic. 

Open-ended questions also help to uncover certainty 
of feelings and expressions of intensity, although well- 
designed closed questions can do the same. 


Thoroughness of Prior Thought If a participant 
has developed a clear opinion on the topic, a closed ques- 
tion does well. If an answer has not been thought out, an 
open-ended question may give the participant a chance to 
ponder a reply, and then elaborate on and revise it. 


Communication Skill Open-ended questions re- 
quire a stronger grasp of vocabulary and a greater ability 
to frame responses than do closed questions. 


Participant Motivation Experience has shown that 
closed questions typically require less motivation and an- 
swering them is less threatening to participants. But the 
response alternatives sometimes suggest which answer is 
appropriate; for this reason, closed questions may be biased. 

While the open-ended question offers many advan- 
tages, closed questions are generally preferable in large 
surveys. They reduce the variability of response, make 
fewer demands on interviewer skills, are less costly to ad- 
minister, and are much easier to code and analyze. After 
adequate exploration and testing, we can often develop 
closed questions that will perform as effectively as open- 
ended questions in many situations. Experimental stud- 
ies suggest that closed questions are equal or superior to 
open-ended questions in many more applications than is 
commonly believed.'* 


Problem/Solution 


Problems: Checklist appears to offer 
options that are neither exhaustive 
nor mutually exclusive. Also, it doesn’t 
fully address the content needs of 
understanding why people choose 

a hotel when they travel for personal 
reasons versus business reasons. 


Solution: Organize the alternatives. 
Create subsets within choices; use 
color or shading to highlight subsets. 
For coding ease, expand the alterna- 
tives so the participant does not fre- 
quently choose “Other.” 


Problems: Double-barreled ques- 
tion; no time frame for the behavior; 
“frequently” is an undefined construct 
for eating behavior; depending on 
the study’s purpose, “order” is not as 
powerful a concept for measurement 
as others (e.g., purchase, consume, 
or eat) 


Solution: Split the questions; expand 
the response alternatives; clearly 
define the construct you want to 
measure. 


Problem: Nonspecific time frame; 
likely to experience memory decay; 
nonspecific screen (not asking what 
you really need to know to qualify a 
participant). 


Solution: Replace “ever” with a more 
appropriate time frame; screen for the 
desired behavior. 


Problem: Question faces serious 
memory decay as a coat may not be 
purchased each year; isn’t asking if 
the coat was a personal purchase or 
for someone else; nor do you know 
the type of coat purchased; nor do 
you know whether the coat was pur- 
chased for full price or at a discount. 


Solution: Limit the time frame; spec- 
ify the coat type. 
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Exhibit 13a-5 Reconstructing Questions 


Poor Measurement Question 


f your purpose for THIS hotel stay 
included personal pleasure, for what 
ONE purpose specifically? 

Visit friend/relative 

Weekend escape 

Sporting event 

Sightseeing 

Family event 

Vacation 

Other: 


uu 


J 
I 
C 
J 
I 
J 


When you eat out, do you frequently 
order appetizers and dessert? 
EYES “4 NO 


Have you ever attended a college 
basketball game? 


Li YES L1 NO 


How much did you pay for the last 
coat you purchased? 


Improved Measurement Question 


Which reason BEST explains your purpose for 

THIS personal pleasure hotel stay? 

LI Dining 

LI Shopping 

LJ Entertainment 
...was this fora... 


(1 Sport-related event? 
LI Theater, musical, or 

other performance? 
[J Museum or exhibit? 


LI Visit friend/relative 
...was this for a special event? LIYES LYNO 
C1 Vacation 
... was this primarily for... 


LI Sightseeing? 
LJ Weekend 
escape? 


LJ Other: 


Considering your personal eating experiences 
away from home in the last 30 days, did you 
purchase an appetizer or dessert more than half 
the time? 


More Than Less Than 
Half the Time — Half the Time 
Purchased 
an appetizer LI O 
a dessert LI im] 


(4 Purchased neither appetizers nor desserts. 


In the last six months, have you been a spectator 
at a basketball game played by college teams on 
a college campus? 


Li YES 4 NO 


Did you purchase a dress coat for your personal 

use in the last GO days? 

O YES 1 NO 
Thinking of this dress coat, how much did 
you pay? (to the nearest dollar) $ .00 
Was this coat purchase made at a 
discounted price? 

EYES M4 NO 
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learningobjectives 

After reading this chapter, you should understand .. . 

The two premises on which sampling theory is based. 

The characteristics of accuracy and precision for measuring sample validity. 
The five questions that must be answered to develop a sampling plan. 


The two categories of sampling techniques and the variety of sampling techniques within each category. 


The various sampling techniques and when each is used. 


6 6We have to hear what’s being said in a 
natural environment, and social media is 
an obvious place to do this, but we also 
have to go and discover the opinions that 


are not being openly shared. Only then can 
we understand the dichotomy between the 
public and private persona. 9 9 


Ben Leet, sales director 
Uhsy-laale) 


bringingresearcnhtolife 


“The ideal participant is thoughtful, articulate, rational, 
and, above all, cooperative. Real people, however, are 
fractious, stubborn, ill-informed, and even perverse. 
Nevertheless, they are who you have to work with,” 
muses Jason as he and Sara hammer out the details of 
the Glacier Symphony sampling plan. 

“Sam Champion, marketing director for CityBus,” 
shares Sara, “certainly had sampling problems. He 
allowed a novice researcher—Eric Burbidge—to 
do the sampling to determine where the company 
could most effectively promote its new daily route 
schedule. Its big problem was a small budget and 
riders from two separate cities, where two different 
papers had substantial circulations—and just as 
substantial advertising rates. CityBus was hoping 
to advertise in only one paper. But the newspapers 
didn’t have circulation figures for specific news 
vending boxes. Champion told the tale at the last 
MRA luncheon. 

“It seems Burbidge was inexperienced enough to 
try to answer CityBus’s question of which newspaper 
to use for ads by conducting a survey on one bus that 
runs between the two cities during evening rush hour. 
Burbidge boards the bus on route 99 and tells the driver 
he’s from headquarters and there to do an official 
survey during the evening ride.” Sara pauses for effect 
and lowers her voice to mimic a base frog. “I need 
to test my hypothesis that readership of newspapers 
on route 99 is equally divided between the East City 
Gazette and the West City Tribune.” 

Jason, now interested, interrupts, “He said that to a 
busload of passengers?” 


“Well, no, the passengers hadn’t yet boarded. The 
way Champion told the story, Burbidge barged his way 
to the front of the line and rapped his clipboard on the 
door to gain entry before any of the passengers could 
board. He said that to the driver. 

“Anyway, Burbidge distributes his questionnaires, 
and the passengers diligently complete them and bring 
them forward to where Burbidge sits at the front of the 
bus. And then they start to play paper-ball hockey in 
the aisle of the bus.” 

“Paper-ball hockey?” questions Jason. 

“Evidently they wad the newspapers they have been 
reading while waiting for the bus into balls and bat them 
through the legs of self-appointed goalies at each end 
of the bus aisle. Anyway, the driver tells Burbidge that 
since East City Club plays hockey that night that when 
he cleans out the bus most of the newspapers will be the 
East City Gazette. The riders evidently like to study the 
night’s pro game in advance, so newsstand sales are brisk 
in the terminal, but only for the newspaper that does the 
better job of covering the sport du jour. Of course the next 
night, the riders would be buying the West City Tribune 
because it does a better job of covering pro basketball. 

“Burbidge is upset and mumbles something about 
the survey asking for the paper most recently purchased. 
The driver tells him not to sweat it. ‘They buy the 
Gazette before hockey and the Trib before basketball 
... but of course in the morning they bring the paper 
that is dropped on their doorstep.’ 

“Burbidge is now mumbling that by choosing route 
99, and choosing hockey night, he has totally distorted 
his results. 


“The driver, who is Champion’s favorite dart 
opponent, is thoroughly enjoying Burbidge’s discomfort 
because he was acting like such an ass at the beginning. 
So the driver tells Burbidge, ‘I know from reading the 
CityBus newsletter that by the time you announce the 
new routes and schedules, we will be finished with 
hockey and basketball and into the baseball season. 
And, of course, most of these folks on the 5:15 bus are 
East City folks, while most on the 5:45 bus are West 
City folks, so your outcome will naturally be affected 
by choosing the 5:15 any time of the year.’ 

“Burbidge, fully exasperated, asks the driver, ‘Is 
there anything else you would care to share with me?’ 

“The driver evidently couldn’t hide his grin when 
he says, “The riders on the 5:45 usually don’t read the 


bringingresearchtolifecont’d 


newspaper much at all. They’ve been watching sports in 
the terminal bar while waiting for the bus. Most aren’t 
feeling any pain—if you get my meaning—and can’t read 
the small newsprint as I don’t turn on the overhead lights.” 

Sara pauses, allowing Jason to ask, “Is there a lesson 
to this story, Sara?” 

“Well, we’ve been talking about having the student 
musicians distribute and collect surveys at each Friday 
evening’s performance. I’m wondering if Glacier 
Symphony has any demographic data from previous 
surveys that might shed some light on concert attendees. 
Td hate to systematically bias our sample, like Burbidge 
did. Since we won’t be present to collect the data—like 
he was—we might never know.” 
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Most people intuitively understand the idea of sampling. One taste from a drink tells us whether it is 
sweet or sour. If we select a few ads from a magazine, we usually assume our selection reflects the 
characteristics of the full set. If some members of our staff favor a promotional strategy, we infer that 
others will also. These examples vary in their representativeness, but each is a sample. 

The basic idea of sampling is that by selecting some of the elements in a population, we may draw 
conclusions about the entire population. A population element is the individual participant or object 
on which the measurement is taken. It is the unit of study. Although an element may be a person, it can 
just as easily be something else. For example, each staff member questioned about an optimal promo- 
tional strategy is a population element, each advertising account analyzed is an element of an account 
population, and each ad is an element of a population of advertisements. A population is the total 
collection of elements about which we wish to make some inferences. All office workers in the firm 
compose a population of interest; all 4,000 files define a population of interest. A census is a count of 
all the elements in a population. If 4,000 files define the population, a census would obtain information 
from every one of them. We call the listing of all population elements from which the sample will be 
drawn the sample frame. 

For CityBus, the population of interest is all riders of affected routes in the forthcoming route re- 
structuring. In studying customer satisfaction with the CompleteCare service operation for MindWriter, 
the population of interest is all individuals who have had a laptop repaired while the CompleteCare 
program has been in effect. The population element is any one individual interacting with the service 
program. 


There are several compelling reasons for sampling, including (1) lower cost, (2) greater accuracy of 
results, (3) greater speed of data collection, and (4) availability of population elements. 


The economic advantages of taking a sample rather than a census are massive. Consider the cost of tak- 
ing acensus. In 2000, due to a Supreme Court ruling requiring a census rather than statistical sampling 
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techniques, the U.S. Bureau of the Census increased its 2000 Decennial Census budget estimate by 
$1.723 billion, to $4.512 billion.’ Is it any wonder that researchers in all types of organizations ask, 
Why should we spend thousands of dollars interviewing all 4,000 employees in our company if we can 
find out what we need to know by asking only a few hundred? 


Greater Accuracy of Results 


Deming argues that the quality of a study is often better with sampling than with a census. He suggests, 
“Sampling possesses the possibility of better interviewing (testing), more thorough investigation of 
missing, wrong, or suspicious information, better supervision, and better processing than is possible 
with complete coverage.”” Research findings substantiate this opinion. More than 90 percent of the 
total survey error in one study was from nonsampling sources and only 10 percent or less was from 
random sampling error.* The U.S. Bureau of the Census, while mandated to take a census of the popula- 
tion every 10 years, shows its confidence in sampling by taking sample surveys to check the accuracy 
of its census. The U.S. Bureau of the Census knows that in a census, segments of the population are 
seriously undercounted. Only when the population is small, accessible, and highly variable is accuracy 
likely to be greater with a census than a sample. 


Greater Speed of Data Collection 


Sampling’s speed of execution reduces the time between the recognition of a need for information and 
the availability of that information. For every disgruntled customer that the MindWriter CompleteCare 
program generates, several prospective customers will move away from MindWriter to a competitor’s 
laptop. So fixing the problems within the CompleteCare program will not only keep current customers 
coming back but also discourage prospective customers from defecting to competitive brands due to 
negative word of mouth. 


Availability of Population Elements 


Some situations require sampling. Safety is a compelling marketing appeal for most vehicles. Yet we 
must have evidence to make such a claim. So we crash-test cars to test bumper strength or efficiency 
of airbags to prevent injury. In testing for such evidence, we destroy the cars we test. A census would 
mean complete destruction of all cars manufactured. Sampling is also the only process possible if the 
population is infinite. 


Sample versus Census 


The advantages of sampling over census studies are less compelling when the population is small and 
the variability within the population is high. Two conditions are appropriate for a census study: a cen- 
sus is (1) feasible when the population is small and (2) necessary when the elements are quite different 
from each other.* When the population is small and variable, any sample we draw may not be repre- 
sentative of the population from which it is drawn. The resulting values we calculate from the sample 
are incorrect as estimates of the population values. Consider North American manufacturers of stereo 
components. Fewer than 50 companies design, develop, and manufacture amplifier and loudspeaker 
products at the high end of the price range. The size of this population suggests a census is feasible. 
The diversity of their product offerings makes it difficult to accurately sample from this group. Some 
companies specialize in speakers, some in amplifier technology, and others in compact-disc transports. 
Choosing a census in this situation is appropriate. 


What Is a Good Sample? 


The ultimate test of a sample design is how well it represents the characteristics of the population it 
purports to represent. In measurement terms, the sample must be valid. Validity of a sample depends 
on two considerations: accuracy and precision. 
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Ford Reenergizes by Changing Sampling Strategy 


In the midst of the financial crisis in the automo- 
bile industry, Ford’s James Farley decided his 
research was excluding a very important sample 
unit: the dealer. With dealers controlling 75 per- 
cent of advertising expenditures for the auto 
giant, Farley thought excluding them as research 
subjects was suicidal. So he recruited 30 of the 
most influential dealers to fly to Detroit to provide 
information and critique the creative proposals 
of the Ford ad agency, Team Detroit. 
Farmington Hills (Ml) full-service research 
firm Morpace put the dealers through an inten- 
sive focus group experience. The dealers were 
soon challenged with questions. “Which incen- 
tives work and which don’t?” “What does the 
Ford brand mean to you?” “What is wrong with 
Ford’s advertising?” In subsequent sessions, 
the dealers were asked to critique ad slogans and branding significant buy-in for the next ad campaign. Farley’s actions 
strategies, recommending those that best capture the Ford gave voice to its dealers with its altered research sampling 
experience. The dealers left the 72-hour marathon session strategy. 
enthusiastic about the direction Ford was taking and with www.ford.com; www.morpace.com; www.teamdetroit.com 


Accuracy 


Accuracy is the degree to which bias is absent from the sample. When the sample is drawn properly, 
the measure of behavior, attitudes, or knowledge (the measurement variables) of some sample elements 
will be /ess than (thus, underestimate) the measure of those same variables drawn from the population. 
Also, the measure of the behavior, attitudes, or knowledge of other sample elements will be more than 
the population values (thus, overestimate them). Variations in these sample values offset each other, 
resulting in a sample value that is close to the population value. For these offsetting effects to occur, 
however, there must be enough elements in the sample, and they must be drawn in a way that favors 
neither overestimation nor underestimation. 

For example, assume you were asked to test the level of brand recall of the “counting sheep” cre- 
ative approach for the Serta mattress company. Hypothetically, you could measure via sample or cen- 
sus. You want a measure of brand recall in combination with message clarity: “Serta mattresses are so 
comfortable you'll feel the difference the minute you lie down.” In 
the census, 52 percent of participants who are TV viewers correctly 
recalled the brand and message. Using a sample, 70 percent recalled 
the brand and correctly interpreted the message. With both results 
for comparison, you would know that your sample was biased, as it 
significantly overestimated the population value of 52 percent. Un- 
fortunately, in most studies taking a census is not feasible, so we 
need an estimate of the amount of error. 

An accurate (unbiased) sample is one in which the underestimators 
offset the overestimators. Systematic variance has been defined as 
“the variation in measures due to some known or unknown influences 
Serta Counting Sheep that ‘cause’ the scores to lean in one direction more than another.’”® 


>chapter 14 Sampling 


Homes on the corner of the block, for example, are often larger and more valuable than those within 
the block. Thus, a sample that selects only corner homes will cause us to overestimate home values in 
the area. Burbidge learned that in selecting bus route 99 for his newspaper readership sample, the time 
of the day, day of the week, and season of the year of the survey dramatically reduced the accuracy and 
validity of his sample. 

Increasing the sample size can reduce systematic variance as a cause of error. However, even the 
large size won’t reduce error if the list from which you draw your participants is biased. The classic 
example of a sample with systematic variance was the Literary Digest presidential election poll in 
1936, in which more than 2 million people participated. The poll predicted Alfred Landon would defeat 
Franklin Roosevelt for the presidency of the United States. Your memory is correct; we’ ve never had a 
president named Alfred Landon. We discovered later that the poll drew its sample from telephone own- 
ers, who were in the middle and upper classes—at the time, the bastion of the Republican Party—while 
Roosevelt appealed to the much larger working class, whose members could not afford to own phones 
and typically voted for the Democratic Party candidate. 


Precision 


A second criterion of a good sample design is precision of estimate. Researchers accept that no sample 
will fully represent its population in all respects. However, to interpret the findings of research, we 
need a measure of how closely the sample represents the population. The numerical descriptors that 
describe samples may be expected to differ from those that describe populations because of random 
fluctuations inherent in the sampling process. This is called sampling error (or random sampling 
error) and reflects the influence of chance in drawing the sample members. Sampling error is what is 
left after all known sources of systematic variance have been accounted for. In theory, sampling error 
consists of random fluctuations only, although some unknown systematic variance may be included 
when too many or too few sample elements possess a particular characteristic. Let’s say Jason draws a 
sample from an alphabetical list of MindWriter owners who are having their laptops currently serviced 
by the CompleteCare program. Assume 80 percent of those surveyed had their laptops serviced by Max 
Jensen. Also assume from the exploratory study that Jensen had more complaint letters about his work 
than any other technician. Arranging the list of laptop owners currently being serviced in an alphabeti- 
cal listing would have failed to randomize the sample frame. If Jason drew the sample from that listing, 
he would actually have increased the sampling error. 

Precision is measured by the standard error of estimate, a type of standard deviation measure- 
ment; the smaller the standard error of estimate, the higher is the precision of the sample. The 
ideal sample design produces a small standard error of estimate. However, not all types of sample 
design provide estimates of precision, and samples of the same size can produce different amounts 
of error. 


Types of Sample Design 


The researcher makes several decisions when designing a sample. These are represented in 
Exhibit 14-1. The sampling decisions flow from two decisions made in the formation of the 
management-research question hierarchy: the nature of the management question and the specific 
investigative questions that evolve from the research question. These decisions are influenced by 
requirements of the project and its objectives, level of risk the researcher can tolerate, budget, time, 
available resources, and culture. 

In the discussion that follows, we will use three examples: 


e The CityBus study introduced in the vignette at the beginning of this chapter. 
¢ The continuing MindWriter CompleteCare customer satisfaction study. 
° A study of the feasibility of starting a dining club near the campus of Metro University. 


The researchers at Metro U are exploring the feasibility of creating a dining club whose facilities 
would be available on a membership basis. To launch this venture, they will need to make a substantial 
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>Exhibit 14-1 Sampling Design within the Research Process 


Management-Research 
Question Hierarchy 


Define Relevant 
Population 


Select Sample Type 
Probability 


Nonprobability 


Select the Sampling 
Technique 


Identify Existing 
Sampling Frames 


Evaluate 
Sampling Frames 


Modify or Construct 
Sampling Frame 


accept 


Probability Select 


Sampling Frame 


Nonprobability 


investment. Research will allow them to reduce many risks. Thus, the research question is, Would 
a membership dining club be a viable enterprise? Some investigative questions that flow from the 
research question include: 


1. 


. How many would join the club under various membership and fee arrangements? 


UA AR WN 


Who would patronize the club, and on what basis? 


. How much would the average member spend per month? 


. What days would be most popular? 


What menu and service formats would be most desirable? 


. What lunch times would be most popular? 


. Given the proposed price levels, how often per month would each member have lunch or 


dinner? 


. What percent of the people in the population say they would join the club, based on the 


projected rates and services? 


We use the last three investigative questions for examples and focus specifically on questions 
7 and 8 for assessing the project’s risks. First, we will digress with other information and examples on 
sample design, coming back to Metro U in the next section. 

In decisions of sample design, the representation basis and the element selection techniques, as 
shown in Exhibit 14-2, classify the different approaches. 
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>Exhibit 14-2 Types of Sampling Designs 


Representation Basis 


Element Selection Probability Nonprobability 


Unrestricted Simple random Convenience 


Representation 


The members of a sample are selected using probability or nonprobability procedures. 

Nonprobability sampling is arbitrary and subjective; when we choose subjectively, we usually do 
so with a pattern or scheme in mind (e.g., only talking with young people or only talking with women). 
Each member of the population does not have a known chance of being included. Allowing interview- 
ers during a mall-intercept study to choose sample elements “at random” (meaning “as they wish” or 
“wherever they find them’) is not random sampling. Although we are not told how Burbidge selected 
the riders of bus route 99 as his sample, it’s clear that he did not use probability sampling techniques. 

Early Internet samples had all the drawbacks of nonprobability samples. Those individuals who 
frequented the Internet were not representative of most target markets or audiences, because far more 
young, technically savvy men frequented the Internet than did any other demographic group. As Inter- 
net use increases and gender discrepancies diminish, many such samples now closely approximate non- 
Internet samples. Of increasing concern, however, is what the Bureau of the Census labels the “great 
digital divide’—low-income and ethnic subgroups’ underrepresentation in their use of the Internet 
compared to the general population. Additionally, many Internet samples were, and still are, drawn 
substantially from panels. These are composed of individuals who have self-selected to become part 
of a pool of individuals interested in participating in online research. There is much discussion among 
professional researchers about whether Internet samples should be treated as probability or nonprob- 
ability samples. Some admit that any sample drawn from a panel is more appropriately treated as a 
nonprobability sample; others vehemently disagree, citing the success of such well-known panels as 
NielsenMedia’s People Meter panels for TV audience assessment and IRI’s BehaviorScan panel for 
tracking consumer packaged goods. As you study the differences here, you should draw your own 
conclusion. 

Key to the difference between nonprobability and probability samples is the term random. In the 
dictionary, random is defined as “without pattern” or as “haphazard.” In sampling, random means 
something else entirely. Probability sampling is based on the concept of random selection—a con- 
trolled procedure that assures that each population element is given a known nonzero chance of se- 
lection. This procedure is never haphazard. Only probability samples provide estimates of precision. 
When a researcher is making a decision that will influence the expenditure of thousands, if not millions, 
of dollars, an estimate of precision is critical. Also, only probability samples offer the opportunity to 
generalize the findings to the population of interest from the sample population. Although exploratory 
research does not necessarily demand this, explanatory, descriptive, and causal studies do. 


Element Selection 


Whether the elements are selected individually and directly from the population—viewed as a single 
pool—or additional controls are imposed, element selection may also classify samples. If each sample 
element is drawn individually from the population at large, it is an unrestricted sample. Restricted sam- 
pling covers all other forms of sampling. 
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Research for Good: Using Charity as an Incentive 


If you could feed a hungry child or prevent the euthanasia of a 
dog by taking a survey, would you? 

Researchers have been using online panels to draw samples 
for Web, email, and mobile surveys for the last decade. Most 
of these panels are developed using three sources: advertising 
networks (sample providers advertise via the Web, email, and 
other media to attract individuals who are willing to participate in 
surveys), loyalty programs (a sponsor company uses its own list 
of individuals who are part of its loyalty efforts and recruit them to 
take surveys) and social media (sample providers use Facebook, 
Twitter, and numerous other social media to recruit participants). 
Sampling firms under the current model employing these sources 
often incentivize their participants with money, Internet currency 
or points, or prizes. Unfortunately, one of the drawbacks is 
a small pool of individuals—even though in the millions—from 
which thousands of firms are drawing their participants. 

The founding partners of Research for Good (RFG) were con- 
cerned that people receiving a personal incentive to give their 
opinion tend to only represent a particular segment of the popu- 
lation. Also, there was the ongoing industry concern about the 
development of “professional respondents”—when volunteer 
participants are recruited by hundreds of sampling companies 
in roughly the same three ways to meet an ever-growing de- 
mand for survey participants. RFG was also concerned about 
social responsibility. So, it developed an incentivized sampling 
model that uses charitable donations to attract uniquely different 
respondents. Its SaySo for Good panel draws on the 90 percent 
of U.S. and Canadian adults who support at least one charity. 
When joining the panel, each participant chooses a charity to 
receive their survey incentive. The RFG database of charities in- 
cludes every government-registered charity in both the U.S. and 
Canada. RFG delivers to the specified charity $1.00 or 25 per- 
cent of the budgeted cost of a per completed survey for each 


Research for Good Sample Composition 


U1 Donation only 
HB Donation + points/cash 
DO Donation + virtual currency 


participant (i.e., if a completed survey is budgeted at $8.00 per 
participant, then the charity receives $2.00). 

RFG has discovered that participants attracted by charity 
incentives are different—in both behavior and attitude—than 
those attracted to a research panel by other means. These sur- 
vey takers tend to be cause-minded, are financial supporters 
or volunteers to charities, and are not likely to be motivated by 
typical cash or prize incentives. Their response rates are higher, 
as well as their completion rates. And they are infrequent survey 
takers, reducing the concern about “professional respondents.” 
RFG now counts thousands of charity-incentivized members in 
its panel. “By attracting new participants, we will generate bet- 
ter data quality while we serve the greater good,” emphasized 
cofounder Sean Case. As a result, RFG now composes its sam- 
ples by including this charity-incentive model. 


www.researchforgood.com; www.saysoforgood.com 
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There are several questions to be answered in securing a sample. Each requires unique information. 
While the questions presented here are sequential, an answer to one question often forces a revision to 
an earlier one. 


1. What is the target population? 

2. What are the parameters of interest? 

3. What is the sampling frame? 

4. What is the appropriate sampling method? 


5. What size sample is needed? 
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What Is the Target Population? 


The definition of the population may be apparent from the management problem or the research 
question(s), but often it is not. Is the population for the dining club study at Metro University defined 
as “full-time day students on the main campus of Metro U’”’? Or should the population include “all 
persons employed at Metro U”? Or should townspeople who live in the neighborhood be included? 
Without knowing the target market chosen for the new venture, it is not obvious which of these is the 
appropriate sampling population. 

There also may be confusion about whether the population consists of individuals, households, or 
families, or a combination of these. If a communication study needs to measure income, then the defini- 
tion of the population element as individual or household can make quite a difference. In an observa- 
tion study, a sample population might be nonpersonal: displays within a store or any ATM a bank owns 
or all single-family residential properties in a community. Good operational definitions are critical in 
choosing the relevant population. 

Assume the Metro University Dining Club is to be solely for the students and employees on the main 
campus. The researchers might define the population as “all currently enrolled students and employees on 
the main campus of Metro U.” However, this does not include family members. They may want to revise 
the definition to make it “current students and employees of Metro U, main campus, and their families.” 

In the nonprobability sample, Burbidge seems to have defined his relevant population as any rider 
of the CityBus system. He presumes he has an equal need to determine newspaper readership of both 
regular and infrequent CityBus riders so that he might reach them with information about the new route 
structure, maps, and schedules. He can, however, easily reach regular riders by distributing information 
about the new routes via display racks on the bus for a period before the new routes are implemented. 
Infrequent riders, then, are the real population of interest of his newspaper readership study. 


What Are the Parameters of Interest? 


Population parameters are summary descriptors (e.g., incidence proportion, mean, variance) of vari- 
ables of interest in the population. Sample statistics are descriptors of those same relevant variables 
computed from sample data. Sample statistics are used as estimators of population parameters. The 
sample statistics are the basis of our inferences about the population. Depending on how measure- 
ment questions are phrased, each may collect a different level of data. Each different level of data also 
generates different sample statistics. Thus, choosing the parameters of interest will actually dictate the 
sample type and its size. 


>Exhibit 14-3 Example Population Parameters 


Population Parameter of Interest Data Level & Measurement Scale 


Ordinal 

(more than 10 times, 6 to 10 times, 
5 or fewer times) 

Ratio 

(absolute number of rides) 


CityBus Frequency of ridership within 7 days 


Metro U Ratio 


(actual eating experiences) 


Frequency of eating on or near campus within 
the last 30 days 

Proportion of students/employees expressing 

interest in dining club 


Nominal 
(interested, not interested) 
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Percent of U.S. Households Accessible by Phone Method of Sampling Invitation 


Cell Phone Only 


32 


Landline Only Neither Cell nor Landline 


11 2 


>picprofile 


Mixed-access sampling means that multiple methods are used to invite participants to a research study—phone, email, 
mobile/wireless, address-based/mail, etc. Approximately 98 percent of possible participants are reachable by phone, whereas 
only 80 percent are reachable online. Mixed-access sampling reduces noncoverage error and nonresponse error. Once a par- 
ticipant is recruited, regardless of the means, he or she may complete the study by a different mode (e.g., recruited by phone 
but take a survey online). Sample recruitment is increasingly done by mixed access. www.surveysampling.com 


Asking Metro U affiliates to reveal their frequency of eating on or near campus (less than 5 times 
per week, greater than 5 but less than 10 times per week, or greater than 10 times per week) would 
provide an ordinal data estimator. Of course, we could ask the question differently and obtain an 
absolute count of eating experiences and that would generate ratio data. In MindWriter, the rating 
of service by CompleteCare on a 5-point scale would be an example of an interval data estimator. 
Asking the CityBus riders about their number of days of ridership during the past seven days would 
result in ratio data. Exhibit 14-3 indicates population parameters of interest for our three example 
studies. 

When the variables of interest in the study are measured on interval or ratio scales, we use the 
sample mean to estimate the population mean and the sample standard deviation to estimate the 
population standard deviation. When the variables of interest are measured on nominal or ordinal 
scales, we use the sample proportion of incidence to estimate the population proportion and the pq 
to estimate the population variance. The population proportion of incidence “is equal to the num- 
ber of elements in the population belonging to the category of interest, divided by the total number 
of elements in the population.’ Proportion measures are necessary for nominal data and are widely 
used for other measures as well. The most frequent proportion measure is the percentage. In the 
Metro U study, examples of nominal data are the proportion of a population that expresses interest 
in joining the club (e.g., 30 percent; therefore p is equal to 0.3 and q, those not interested, equals 
0.7) or the proportion of married students who report they now eat in restaurants at least five times 
a month. The CityBus study seeks to determine whether East City or West City has the most rid- 
ers on bus route 99. MindWriter might want to know if men or women have experienced the most 
problems with laptop model 9000. These measures for CityBus and MindWriter would result in 
nominal data. 

There may also be important subgroups in the population about whom we would like to make 
estimates. For example, we might want to draw conclusions about the extent of dining club use that 
could be expected from married students versus single students, residential students versus commuter 
students, and so forth. Such questions have a strong impact on the nature of the sampling frame we 
accept (we would want the list organized by these subgroups, or within the list each characteristic of 
each element would need to be noted), the design of the sample, and its size. Burbidge should be more 
interested in reaching infrequent rather than regular CityBus riders with the newspaper advertising 
he plans; to reach frequent riders CityBus could use on-bus signs or distribute paper schedules rather 
than using more expensive newspaper ads. And in the MindWriter study, Jason may be interested in 
comparing the responses of those who experienced poor service and those who experienced excellent 
service through the CompleteCare program. 
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What Is the Sampling Frame? 


The sampling frame is closely related to the population. It is the list of elements from which the sample 
is actually drawn. Ideally, it is a complete and correct list of population members only. Jason should 
find limited problems obtaining a sampling frame of CompleteCare service users, as MindWriter has 
maintained a database of all calls coming into the call center and all serial numbers of laptops serviced. 

As a practical matter, however, the sampling frame often differs from the theoretical population. For 
the dining club study, the Metro U directory would be the logical first choice as a sampling frame. Direc- 
tories are usually accurate when published in the fall, but suppose the study is being done in the spring. 
The directory will contain errors and omissions because some people will have withdrawn or left since 
the directory was published, while others will have enrolled or been hired. Usually university directo- 
ries don’t mention the families of students or employees. Just how much inaccuracy one can tolerate in 
choosing a sampling frame is a matter of judgment. You might use the directory anyway, ignoring the 
fact that it is not a fully accurate list. However, if the directory is a year old, the amount of error might 
be unacceptable. One way to make the sampling frame for the Metro U study more representative of the 
population would be to secure a supplemental list of the new students and employees as well as a list of 
the withdrawals and terminations from Metro U’s registrar and human resources databases. You could 
then add and delete information from the original directory. Or, if their privacy policies permit, you might 
just request a current listing from each of these offices and use these lists as your sampling frame. 

A greater distortion would be introduced if a branch campus population were included in the Metro 
U directory. This would be an example of a too inclusive frame—that is, a frame that includes many 
elements other than the ones in which we are interested. A university directory that includes faculty and 
staff retirees is another example of a too inclusive sampling frame. 

Often you have to accept a sampling frame that includes people or cases beyond those in whom you 
are interested. You may have to use a telephone directory to draw a sample of business telephone num- 
bers. Fortunately, this is easily resolved. You draw a sample from the larger population and then use a 
screening procedure to eliminate those who are not members of the group you wish to study. 

The Metro U dining club survey is an example of a sampling frame problem that is readily solved. 
Often one finds this task much more of a challenge. Suppose you need to sample the members of 
an ethnic group, say, Asians residing in Little 
Rock, Arkansas. There is probably no directory 
of this population. Although you may use the 
general city directory, sampling from this too 
inclusive frame would be costly and inefficient, 
because Asians represent only a small fraction 
of Little Rock’s population. The screening task 
would be monumental. Since ethnic groups fre- 
quently cluster in certain neighborhoods, you 
might identify these areas of concentration and 
then use a reverse area telephone or city direc- 
tory, which is organized by street address, to 
draw the sample. Burbidge had a definite prob- 
lem, because no sample frame of CityBus rid- 
ers existed. Although some regular riders used 
monthly passes, infrequent riders usually paid 
cash for their fares. It might have been possible 
for Burbidge to anticipate this and to develop 
over time a listing of customers. Bus drivers 
could have collected relevant contact informa- 
tion over a month, but the cost of contacting 
customers via phone or mail would have been 
much more expensive than the self-administered 
intercept approach Burbidge chose for data col- 
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A decade ago, Chinese families with a home phone were envied. By February 
: : : 2012, China's cellular telephone users exceeded 1 billion, within a population 
lection. One sampling frame available to Bur-  6F 4.8 billion. During such a period of rapid growth, business or personal phone 
bidge was a list of bus routes. This list would _ |istings are inadequate as a sampling frame. 
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have allowed him to draw a probability sample using a cluster sampling technique. We discuss more 
complex sampling techniques later in this chapter. 

The sampling issues we have discussed so far are fairly universal. It is not until we begin talking 
about sampling frames and sampling methods that international research starts to deviate. International 
researchers often face far more difficulty in locating or building sample frames. Countries differ in 
how each defines its population; this affects census and relevant population counts.’ Some countries 
purposefully oversample to facilitate the analysis of issues of particular national interest; this means we 
need to be cautious in interpreting published aggregate national figures.’ These distinctions and diffi- 
culties may lead the researcher to choose nonprobability techniques or different probability techniques 
than they would choose if doing such research in the United States or other developed countries. In a 
study that is fielded in numerous countries at the same time, researchers may use different sampling 
methodologies, resulting in hybrid studies that will need care to be combined. It is common practice to 
weight sample data in cross-national studies to develop sample data that are representative.'° Choice 
of sampling methods is often dictated by culture as much as by communication and technology in- 
frastructure. Just as all advertising campaigns would not be appropriate in all parts of the world, all 
sampling techniques would not be appropriate in all subcultures. Our discussion in this text focuses 
more on domestic than international research. We believe it is easier to learn the principles of research 
in an environment that you know versus one in which many students can only speculate. Yet we also 
believe that ethnic and cultural sensitivity should influence every decision of researchers, whether they 
do research domestically or internationally. 


What Is the Appropriate Sampling Method? 


The researcher faces a basic choice: a probability or nonprobability sample. With a probability sam- 
ple, a researcher can make probability-based confidence estimates of various parameters that cannot 
be made with nonprobability samples. Choosing a probability sampling technique has several conse- 
quences. A researcher must follow appropriate procedures so that: 


e Interviewers or others cannot modify the selections made. 
° Only the selected elements from the original sampling frame are included. 


e Substitutions are excluded except as clearly specified and controlled according to predetermined 
decision rules. 


Despite all due care, the actual sample achieved will not match perfectly the sample that is originally 
drawn. Some people will refuse to participate, and others will be difficult, if not impossible, to find. 
Thus, no matter how careful we are in replacing those who refuse or are never located, sampling error 
is likely to rise. 

With personnel records available at a university and a population that is geographically concen- 
trated, a probability sampling method is possible in the dining club study. University directories are 
generally available, and the costs of using a simple random sample would not be great here. Then, too, 
since the researchers are thinking of a major investment in the dining club, they would like to be highly 
confident they have a representative sample. The same analysis holds true for MindWriter: A sample 
frame is readily available, making a probability sample possible and likely. 

Although the probability cluster sampling technique was available to him, it is obvious that Burbidge 
chose nonprobability sampling, arbitrarily choosing bus route 99 as a judgment sample and attempting 
to survey everyone riding the bus during the arbitrary times in which he chose to ride. What drove him 
to this decision is likely what makes researchers turn to nonprobability sampling in other situations: 
ease, speed, and cost. 


What Size Sample Is Needed? 


Much folklore surrounds this question. The most pervasive myths are (1) a sample must be large or 
it is not representative and (2) a sample should bear some proportional relationship to the size of the 
population from which it is drawn. With nonprobability samples, researchers confirm these myths 
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using the number of subgroups, rules of thumb, and budget considerations to settle on a sample size. 
In probability sampling, how large a sample should be is a function of the variation in the population 
parameters under study and the estimating precision needed by the researcher. Some principles that 
influence sample size include: 


e The greater the dispersion or variance within the population, the larger the sample must be to 
provide estimation precision. 


¢ The greater the desired precision of the estimate, the larger the sample must be. 
e The narrower or smaller the error range, the larger the sample must be. 
¢ The higher the confidence level in the estimate, the larger the sample must be. 


¢ The greater the number of subgroups of interest within a sample, the greater the sample size 
must be, as each subgroup must meet minimum sample size requirements. 


Cost considerations influence decisions about the size and type of sample and the data collection 
methods. Almost all studies have some budgetary constraint, and this may encourage a researcher to 
use a nonprobability sample. Probability sample surveys incur list costs for sample frames, callback 
costs, and a variety of other costs that are not necessary when nonprobability samples are used. But 
when the data collection method is changed, the amount and type of data that can be obtained also 
change. Note the effect of a $2,000 budget on sampling considerations: 


e Simple random sampling: $25 per interview; 80 completed interviews. 
e Geographic cluster sampling: $20 per interview; 100 completed interviews. 
¢ Self-administered questionnaire: $12 per respondent; 167 completed instruments. 


¢ Telephone interviews: $10 per respondent; 200 completed interviews."' 


For CityBus the cost of sampling riders’ newspaper preferences to discover where to run the 
route-reconfiguration announcements must be significantly less than the cost of running ads in 
both East City and West City dailies. Thus, the nonprobability judgment sampling procedure that 
Burbidge used was logical from a budget standpoint. The investment required to open the dining 
club at Metro U also justifies the more careful probability approach taken by the students. For 
MindWriter, an investment in CompleteCare has already been made; Jason needs to be highly con- 
fident that his recommendations to change CompleteCare procedures and policies are on target and 
thoroughly supported by the data collected. These considerations justify MindWriter’s probability 
sampling approach. 


> Probability Sampling 
Simple Random Sampling 


The unrestricted, simple random sample is the purest form of probability sampling. Since all probabil- 
ity samples must provide a known nonzero probability of selection for each population element, the 
simple random sample is considered a special case in which each population element has a known 
and equal chance of selection. 


Sample size 


Probability of selection = Population size 


The Metro U dining club study has a population of 20,000. If the sample size is 300, the probability of 
selection is 1.5 percent (300/20,000 = 0.015). In this section, we use the simple random sample to build 
a foundation for understanding sampling procedures and choosing probability samples. The simple 
random sample is easy to implement with automatic dialing (random dialing) and with computerized 
voice response systems. However, it requires a list of population elements, can be time-consuming and 
expensive, and can require larger sample sizes than other probability methods. Exhibit 14-4 provides 
an overview of the steps involved in choosing a random sample. 
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>Exhibit 14-4 How to Choose a Random Sample 


Selecting a random sample is accomplished with the aid of computer software, a table of random numbers, or a calculator with a 
random number generator. Drawing slips out of a hat or Ping-Pong balls from a drum serves as an alternative if every element in the 
sampling frame has an equal chance of selection. Mixing the slips (or balls) and returning them between every selection ensures that 
every element is just as likely to be selected as any other. 


A table of random numbers (such as Appendix D, Exhibit D-10) is a practical solution when no software program is available. 


Random number tables contain digits that have no systematic organization. Whether you look at rows, columns, or diagonals, you 
will find neither sequence nor order. Exhibit C-10 in Appendix C is arranged into 10 columns of five-digit strings, but this is solely for 


readability. 
Assume the researchers want a sample of 10 from a population of 95 elements. How will the researcher begin? 


1. Assign each element within the sampling frame a unique number from 01 to 95. 


8. Determine how the digits in the random number table will be assigned to the sampling frame to choose the specified sample size 


(researchers agree to read the first two digits in this column downward until 10 are selected). 


Other approaches to selecting digits are endless: horizontally right to left, bottom to top, diagonally across columns, and so forth. 
Computer selection of a simple random sample will be more efficient for larger projects. 


Complex Probability Sampling 


Simple random sampling is often impractical. Reasons include (1) it requires a population list (sam- 
pling frame) that is often not available; (2) it fails to use all the information about a population, thus 
resulting in a design that may be wasteful; and (3) it may be expensive to implement in both time and 
money. These problems have led to the development of alternative designs that are superior to the 
simple random design in statistical and/or economic efficiency. 

A more efficient sample in a statistical sense is one that provides a given precision (standard 
error of the mean or proportion) with a smaller sample size. A sample that is economically more 
efficient is one that provides a desired precision at a lower dollar cost. We achieve this with designs 
that enable us to lower the costs of data collecting, usually through reduced travel expense and 
interviewer time. 

In the discussion that follows, four alternative probability sampling approaches are considered: 
(1) systematic sampling, (2) stratified sampling, (3) cluster sampling, and (4) double sampling. 


Systematic Sampling 


A versatile form of probability sampling is systematic sampling. In this approach, every kth ele- 
ment in the population is sampled, beginning with a random start of an element in the range of 1 
to k. The kth element, or skip interval, is determined by dividing the sample size into the population 
size to obtain the skip pattern applied to the sampling frame. This assumes that the sample frame is 
an accurate list of the population; if not, the number of elements in the sample frame is substituted 
for population size. 


Population size 


k = Skip interval = Sample size 


The major advantage of systematic sampling is its simplicity and flexibility. It is easier to instruct field 
workers to choose the dwelling unit listed on every kth line of a listing sheet than it is to use a random 
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numbers table. With systematic sampling, there is no need to number the entries in a large personnel 
file before drawing a sample. To draw a systematic sample, do the following: 


¢ Identify, list, and number the elements in the population. 
¢ Identify the skip interval (k). 
¢ Identify the random start. 


e Draw a sample by choosing every kth entry. 


Invoices or customer accounts can be sampled by using the last digit or a combination of digits of an 
invoice or customer account number. Time sampling is also easily accomplished. Systematic sampling 
would be an appropriate technique for MindWriter’s CompleteCare program evaluation. 

Systematic sampling can introduce subtle biases. A concern with systematic sampling is the pos- 
sible periodicity in the population that parallels the sampling ratio. In sampling restaurant sales of des- 
sert by drawing days of the year, a skip interval of 7 would bias results, no matter which day provides 
the random start. A less obvious case might involve a survey in an area of apartment buildings where 
the typical pattern is eight apartments per building. A skip interval of 8 could easily oversample some 
types of apartments and undersample others. 

Another difficulty may arise when there is a monotonic trend in the population elements. That is, 
the population list varies from the smallest to the largest element or vice versa. Even a chronological 
list may have this effect if a measure has trended in one direction over time. Whether a systematic 
sample drawn under these conditions provides a biased estimate of the population mean or proportion 
depends on the initial random draw. Assume that a list of 2,000 commercial banks is created, arrayed 
from the largest to the smallest, from which a sample of 50 must be drawn for analysis. A skip interval 
of 40 beginning with a random start at 16 would exclude the 15 largest banks and give a small-size bias 
to the findings. 

The only protection against these subtle biases is constant vigilance by the researcher. Some ways 
to avoid such bias include: 


e Randomize the population before sampling (e.g., order the banks by name rather than size). 
e Change the random start several times in the sampling process. 


¢ Replicate a selection of different samples. 


Although systematic sampling has some theoretical problems, from a practical point of view it is 
usually treated as a simple random sample. When similar population elements are grouped within the 
sampling frame, systematic sampling is statistically more efficient than a simple random sample. This 
might occur if the listed elements are ordered chronologically, by size, by class, and so on. Under these 
conditions, the sample approaches a proportional stratified sample. The effect of this ordering is more 
pronounced on the results of cluster samples than for element samples and may call for a proportional 
stratified sampling formula.'” 


Stratified Sampling 


Most populations can be segregated into several mutually exclusive subpopulations, or strata. The 
process by which the sample is constrained to include elements from each of the segments is called 
stratified random sampling. University students can be divided by their class level, school or major, 
gender, and so forth. After a population is divided into the appropriate strata, a simple random sample 
can be taken within each stratum. The results from the study can then be weighted (based on the propor- 
tion of the strata to the population) and combined into appropriate population estimates. 

There are three reasons a researcher chooses a stratified random sample: (1) to increase a sample’s 
statistical efficiency, (2) to provide adequate data for analyzing the various subpopulations or strata, 
and (3) to enable different research methods and procedures to be used in different strata.'° 

Stratification is usually more efficient statistically than simple random sampling and at worst it is 
equal to it. With the ideal stratification, each stratum is homogeneous internally and heterogeneous 
with other strata. This might occur in a sample that includes members of several distinct ethnic groups. 
In this instance, stratification makes a pronounced improvement in statistical efficiency. 
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Keynote Systems Tests the Power of Search 


Twice yearly Keynote Systems evaluates the performance 
of five search engines, including market leader Google, AOL 
Search, Yahoo! Search, Ask.com, and MSN Search. Keynote, 
a “worldwide leader in services that improve online business 
performance and communications technologies,” uses an on- 
line panel to perform “interactive Web site tests to assess user 
experience,” profiling not only how people use search engines, 
but why they search as they do. Keynote allocates participants 
and experimental treatments as in Exhibit 14-5: 2,000 people 
are randomly drawn from more than 160,000 panel members 
and invited to participate via e-mail. They are assigned randomly 
to five groups of 400; each group is assigned a particular search 
engine. Whether participants have any experience with that par- 
ticular engine is not a criterion for assignment. Each group is 
assigned a series of search tasks, starting with a general task— 
Think about anything you would like to search for; go and search 
that—to more specific tasks —find a local establishment, a prod- 
uct, an image, and a news item. Each search engine-allocated 
group essentially performs the same series of tasks. From their 
activities, Keynote generates 250,000 metrics (including time in- 
volved in the search, whether the search was successful, etc.). 
It matches these metrics to survey data used to measure satis- 
faction, perceived difficulty, and specific frustrations. From this 
combined data it develops several indices. 

“One of the things we noted from a series of such tests was 
that Google repeatedly received rave reviews, even in instances 
where performance measures told a different story,” shares se- 
nior research consultant Lance Jones. With almost 60 percent 


market share, Google has strong recognition and tends to set 
the bar in search site design. Is its brand that powerful that it 
can influence attitudes even in the face of conflicting perfor- 
mance experience? If the brand is not a factor, which search 
engine would produce the most satisfying and useful results, the 
best sponsored results, and the best presentation and design? 
Keynote wanted to design an experiment that would show the 
power of the search engine brand. To do that, they needed to 
remove brand identity from the search results. Its solution was 
to design a generic-appearing search engine website and re- 
sults format page, feeding actual search results into its generic 
format. 

For the brand power test (Exhibit 14-6) 2,000 participants 
were again divided into five groups and assigned one search 
engine. This time, however, half the participants were assigned 
to a branded group (n = 200) and would see the results with a 
text line “Results brought to you by Yahoo/Google/Ask, etc.”; 
the other half (n = 200) would see the same results but with- 
out the brand notation line (n = 200). All five search engines 
were tested using the tasks performed in the standard twice- 
annual test, but all the results seen by participants were ac- 
tually generated using the assigned search engine, then fed 
into the generic results presentation. “The results pages were 
delivered live and participants would have perceived no differ- 
ence in elapsed time, as the results were delivered within mil- 
liseconds of what the standard search would have delivered,” 
explained Jones. The test produced 1,600 queries that gener- 
ated 12 distinct metrics. 


>Exhibit 14-5 Participant Allocation in Search Engine Test 


Experiment Participants 
(n = 2000) 


Search Engine A Group 
(n = 400) 


Search Engine B Group 
(n = 400) 


Search Engine C Group 
(n = 400) 


Search Engine D Group 
(n = 400) 


Search Engine E Group 
(n = 400) 


Movie 
(n = 200) 


Grocery 
(n = 200) 


Restaurant 
(n = 200) 


Self-Select 
(n = 200) 
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>Exhibit 14-6 Participant Allocation in Brand Power Test 


Experiment Participants 
(n = 2000) 
Grocery 
Search Engine A Group (n = 200) 
(n = 400) 
|_| Restaurant 
Search Engine B Group (n = 200) 
(n = 400) 
Hoel = 20 
Search Engine C Group ii 200) 
i= 800) Self-Select 
(n = 200) 
Search Engine D Group 
: = ‘re | 
Search Engine E Group 
(n = 400) Grocery 
(n = 200) 
Restaurant 
(n = 200) 
Movie 
(n = 200) 
Self-Select 
(n = 200) 
Is a brand powerful? Here are some sample results for delivered an 800; Google’s sponsored results satisfaction was 
Google; keep in mind that the branded group and the un- 768 (unbranded) compared to 809 (branded); full design satis- 


branded group saw the exact same results pages. On the un- faction was 758 (unbranded) compared to 806 (branded). Evalu- 
branded group, the calculated Google results satisfaction score ate the design of this sample. 


was 782 (on a 1,000-point scale), while the branded group www.keynote.com 


It is also useful when the researcher wants to study the characteristics of certain population subgroups. 
Thus, if one wishes to draw some conclusions about activities in the different classes of a student body, 
stratified sampling would be used. Similarly, if a restaurant were interested in testing menu changes to 
attract younger patrons while retaining its older, loyal customers, stratified sampling using age and prior 
patronage as descriptors would be appropriate. Stratification is also called for when different methods 
of data collection are applied in different parts of the population, a research design that is becoming in- 
creasingly common. This might occur when we survey company employees at the home office with one 
method but must use a different approach with employees scattered throughout the country. 

If data are available on which to base a stratification decision, how shall we go about it?'* The ideal 
stratification would be based on the primary variable under study. If the major concern were to learn 
how often per month patrons would use the Metro U dining club, then one would like to stratify on this 
expected number of use occasions. The only difficulty with this idea is that if we knew this informa- 
tion, we would not need to conduct the study. We must, therefore, pick a variable for stratifying that we 
believe will correlate with the frequency of club use per month, something like days at work or class 
schedule as an indication of when a sample element might be near campus at mealtimes. 

Researchers often have several important variables about which they want to draw conclusions. A 
reasonable approach is to seek some basis for stratification that correlates well with the major variables. 
It might be a single variable (class level), or it might be a compound variable (class by gender). In 
any event, we will have done a good stratifying job if the stratification base maximizes the difference 
among strata means and minimizes the within-stratum variances for the variables of major concern. 
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The more strata used, the closer you come to maximizing interstrata differences (differences be- 
tween strata) and minimizing intrastratum variances (differences within a given stratum). You must 
base the decision partially on the number of subpopulation groups about which you wish to draw sepa- 
rate conclusions. Costs of stratification also enter the decision. The more strata you have, the higher the 
cost of the research project due to the cost associated with more detailed sampling. There is little to be 
gained in estimating population values when the number of strata exceeds six.'* 

The size of the strata samples is calculated with two pieces of information: (1) how large the total 
sample should be and (2) how the total sample should be allocated among strata. In deciding how to 
allocate a total sample among various strata, there are proportionate and disproportionate options. 


Proportionate versus Disproportionate Sampling In proportionate stratified sampling, 
each stratum is properly represented so that the sample size drawn from the stratum is proportionate to 
the stratum’s share of the total population. This approach is more popular than any of the other strati- 
fied sampling procedures. Some reasons for this include: 


° It has higher statistical efficiency than a simple random sample. 
e It is much easier to carry out than other stratifying methods. 


¢ It provides a self-weighting sample; the population mean or proportion can be estimated simply 
by calculating the mean or proportion of all sample cases, eliminating the weighting of responses. 


On the other hand, proportionate stratified samples often gain little in statistical efficiency if the 
strata measures and their variances are similar for the major variables under study. 

Any stratification that departs from the proportionate relationship is disproportionate stratified 
sampling. There are several disproportionate allocation schemes. One type is a judgmentally deter- 
mined disproportion based on the idea that each stratum is large enough to secure adequate confidence 
levels and error range estimates for individual strata. The following table shows the relationship between 
proportionate and disproportionate stratified sampling. 


Proportionate Disproportionate 


Stratum Population Sample Sample 


Male 45% 45% 35% 


A researcher makes decisions regarding disproportionate sampling, however, by considering how a 
sample will be allocated among strata. One author states, 


In a given stratum, take a larger sample if the stratum is larger than other strata; the stratum is more variable internally; 
and sampling is cheaper in the stratum. '® 


If one uses these suggestions as a guide, it is possible to develop an optimal stratification scheme. When 
there is no difference in intrastratum variances and when the costs of sampling among strata are equal, 
the optimal design is a proportionate sample. 

While disproportionate sampling is theoretically superior, there is some question as to whether it has 
wide applicability in a practical sense. If the differences in sampling costs or variances among strata are 
large, then disproportionate sampling is desirable. It has been suggested that “differences of several- 
fold are required to make disproportionate sampling worthwhile.””"” 


The process for drawing a stratified sample is: 


e Determine the variables to use for stratification. 

e Determine the proportions of the stratification variables in the population. 

¢ Select proportionate or disproportionate stratification based on project information needs and risks. 
e Divide the sampling frame into separate frames for each stratum. 

e Randomize the elements within each stratum’s sampling frame. 


¢ Follow random or systematic procedures to draw the sample from each stratum. 
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>Exhibit 14-7 Comparison of Stratified and Cluster Sampling 


Stratified Sampling Cluster Sampling 


1. We divide the population into a few subgroups. 1. We divide the population into many subgroups. 
« Each subgroup has many elements in it. « Each subgroup has few elements in it. 
¢ Subgroups are selected according to some criterion e Subgroups are selected according to some criterion of ease or 

that is related to the variables under study. availability in data collection. 

2. We try to secure homogeneity within subgroups. 2. We try to secure heterogeneity within subgroups. 

8. We try to secure heterogeneity between subgroups. 8. We try to secure homogeneity between subgroups. 

4. We randomly choose elements from within each 4. We randomly choose several subgroups that we then typically 
subgroup. study in depth. 

Cluster Sampling 


In a simple random sample, each population element is selected individually. The population can 
also be divided into groups of elements with some groups randomly selected for study. This is 
cluster sampling. Cluster sampling differs from stratified sampling in several ways, as indicated in 
Exhibit 14-7. 

Two conditions foster the use of cluster sampling: (1) the need for more economic efficiency than 
can be provided by simple random sampling and (2) the frequent unavailability of a practical sampling 
frame for individual elements. 

Statistical efficiency for cluster samples is usually lower than for simple random samples chiefly 
because clusters often don’t meet the need for heterogeneity and, instead, are homogeneous. For 
example, families in the same block (a typical cluster) are often similar in social class, income level, 
ethnic origin, and so forth. Although statistical efficiency in most cluster sampling may be low, 
economic efficiency is often great enough to overcome this weakness. The criterion, then, is the net 
relative efficiency resulting from the trade-off between economic and statistical factors. It may take 
690 interviews with a cluster design to give the same precision as 424 simple random interviews. But 


A low-cost, 
frequently used 
method, the 
area cluster 
sample may 

use geographic 
sample units 
(e.g., city blocks). 
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if it costs only $5 per interview in the cluster situation and $10 in the simple random case, the cluster 
sample is more attractive ($3,450 versus $4,240). 


Area Sampling Much research involves populations that can be identified with some geographic 
area. When this occurs, it is possible to use area sampling, the most important form of cluster sampling. 
This method overcomes the problems of both high sampling cost and the unavailability of a practical sam- 
pling frame for individual elements. Area sampling methods have been applied to national populations, 
county populations, and even smaller areas where there are well-defined political or natural boundaries. 
Suppose you want to survey the adult residents of a city. You would seldom be able to secure a listing 
of such individuals. It would be simple, however, to get a detailed city map that shows the blocks of the 
city. If you take a sample of these blocks, you are also taking a sample of the adult residents of the city. 


Design In designing cluster samples, including area samples, we must answer several questions: 
. How homogeneous are the resulting clusters? 

. Shall we seek equal-size or unequal-size clusters? 

. How large a cluster shall we take? 


. Shall we use a single-stage or multistage cluster? 


An WN Re 


. How large a sample is needed? 


1. When clusters are homogeneous, this contributes to low statistical efficiency. Sometimes one 
can improve this efficiency by constructing clusters to increase intracluster variance. In the dining 
club study, researchers might have chosen a course as a cluster, choosing to sample all students in 
that course if it enrolled students of all four class years. Or maybe they could choose a departmental 
office that had faculty, staff, and administrative positions as well as student workers. In area sampling 
to increase intracluster variance, researchers could combine into a single cluster adjoining blocks that 
contain different income groups or social classes. 


2. A cluster sample may be composed of clusters of equal or unequal size. The theory of cluster- 
ing is that the means of sample clusters are unbiased estimates of the population mean. This is more 
often true when clusters are naturally equal, such as households in city blocks. While one can deal with 
clusters of unequal size, it may be desirable to reduce or counteract the effects of unequal size. There 
are several approaches to this: 


e Combine small clusters and split large clusters until each approximates an average size. 
e Stratify clusters by size and choose clusters from each stratum. 


° Stratify clusters by size and then subsample, using varying sampling fractions to secure an 
overall sampling ratio." 


3. There is no a priori answer to the ideal cluster size question. Comparing the efficiency of differing 
cluster sizes requires that we discover the different costs for each size and estimate the different variances 
of the cluster means. Even with single-stage clusters (where the researchers interview or observe every 
element within a cluster), it is not clear which size (say, 5, 20, or 50) is superior. Some have found that in 
studies using single-stage area clusters, the optimal cluster size is no larger than the typical city block." 


4. Concerning single-stage or multistage cluster designs, for most large-scale area sampling, the 
tendency is to use multistage designs. Several situations justify drawing a sample within a cluster, in 
preference to the direct creation of smaller clusters and taking a census of that cluster using one-stage 
cluster sampling:”° 


e Natural clusters may exist as convenient sampling units yet, for economic reasons, may be larger 
than the desired size. 


e We can avoid the cost of creating smaller clusters in the entire population and confine subsam- 
pling to only those large natural clusters. 


e The sampling of naturally compact clusters may present practical difficulties. For example, inde- 
pendent interviewing of all members of a household may be impractical. 


5. The answer to how many subjects must be interviewed or observed depends heavily on the specific 
cluster design, and the details can be complicated. Unequal clusters and multistage samples are the chief 
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complications, and their statistical treatment is beyond the scope of this book.”! Here we will treat only sin- 
gle-stage sampling with equal-size clusters (called simple cluster sampling). It is analogous to simple ran- 
dom sampling. We can think of a population as consisting of 20,000 clusters of one student each, or 2,000 
clusters of 10 students each, and so on. Assuming the same specifications for precision and confidence, we 
should expect that the calculation of a probability sample size would be the same for both clusters. 


Double Sampling 


It may be more convenient or economical to collect some information by sample and then use this 
information as the basis for selecting a subsample for further study. This procedure is called double 
sampling, sequential sampling, or multiphase sampling. It is usually found with stratified and/or 
cluster designs. The calculation procedures are described in more advanced texts. 

Double sampling can be illustrated by the dining club example. You might use a telephone survey 
or another inexpensive survey method to discover who would be interested in joining such a club and 
the degree of their interest. You might then stratify the interested respondents by degree of interest 
and subsample among them for intensive interviewing on expected consumption patterns, reactions to 
various services, and so on. Whether it is more desirable to gather such information by one-stage or 
two-stage sampling depends largely on the relative costs of the two methods. 

Because of the wide range of sampling designs available, it is often difficult to select an approach 
that meets the needs of the research question and helps to contain the costs of the project. To help 
with these choices, Exhibit 14-8 may be used to compare the various advantages and disadvantages 


>Exhibit 14-8 Comparison of Probability Sampling Designs 


Disadvantages 


Description 


Advantages 


Requires a listing of population 
elements. 

Takes more time to implement. 

Uses larger sample sizes. 

Produces larger errors. 


Simple Random 
Cost: High 
Use: Moderate 


Each population element has 
an equal chance of being 
selected into the sample. 

Sample drawn using random 

number table/generator. 


Easy to implement with automatic dialing 
(random-digit dialing) and with com- 
puterized voice response systems. 


Increased error will result if 
subgroups are selected at 
different rates. 

Especially expensive if strata 

on the population have to be 

created. 


Stratified 
Cost: High 
Use: Moderate 


Divides population into sub- 
populations or strata and 
uses simple random on 
each stratum. Results may 

be weighted and combined. 


Researcher controls sample size in 
strata. 

Increased statistical efficiency. 

Provides data to represent and 
analyze subgroups. 

Enables use of different methods in strata. 


Increased costs if indiscrimi- 
nately used. 


Double 
(Sequential or 
multiphase) 

Cost: Moderate 

Use: Moderate 


Process includes collecting 
data from a sample using 
a previously defined tech- 
nique. Based on the infor- 
mation found, a subsample 
is selected for further study. 


May reduce costs if first stage results in 
enough data to stratify or cluster the 
population. 
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of probability sampling. Nonprobability sampling techniques are covered in the next section. They 
are used frequently and offer the researcher the benefit of low cost. However, they are not based on a 
theoretical framework and do not operate from statistical theory; consequently, they produce selection 
bias and nonrepresentative samples. Despite these weaknesses, their widespread use demands their 
mention here. 


> Nonprobability Sampling 


Any discussion of the relative merits of probability versus nonprobability sampling clearly shows the 
technical superiority of the former. In probability sampling, researchers use a random selection of ele- 
ments to reduce or eliminate sampling bias. Under such conditions, we can have substantial confidence 
that the sample is representative of the population from which it is drawn. In addition, with probability 
sample designs, we can estimate an error range within which the population parameter is expected to 
fall. Thus, we can reduce not only the chance for sampling error but also estimate the range of probable 
sampling error present. 

With a subjective approach like nonprobability sampling, the probability of selecting population 
elements is unknown. There are a variety of ways to choose persons or cases to include in the sample. 
Often we allow the choice of subjects to be made by field workers on the scene. When this occurs, 
there is greater opportunity for bias to enter the sample selection procedure and to distort the findings of 
the study. Also, we cannot estimate any range within which to expect the population parameter. Given 
the technical advantages of probability sampling over nonprobability sampling, why would anyone 
choose the latter? There are some practical reasons for using the less precise methods. 


Practical Considerations 


We may use nonprobability sampling procedures because they satisfactorily meet the sampling objec- 
tives. Although a random sample will give us a true cross section of the population, this may not be 
the objective of the research. If there is no desire or need to generalize to a population parameter, then 
there is much less concern about whether the sample fully reflects the population. Often researchers 
have more limited objectives. They may be looking only for the range of conditions or for examples of 
dramatic variations. This is especially true in exploratory research in which one may wish to contact 
only certain persons or cases that are clearly atypical. Burbidge would have likely wanted a prob- 
ability sample if the decision resting on the data was the actual design of the new CityBus routes and 
schedules. However, the decision of where and when to place advertising announcing the change is a 
relatively low-cost one in comparison. 

Additional reasons for choosing nonprobability over probability sampling are cost and time. Prob- 
ability sampling clearly calls for more planning and repeated callbacks to ensure that each selected 
sample member is contacted. These activities are expensive. Carefully controlled nonprobability sam- 
pling often seems to give acceptable results, so the investigator may not even consider probability sam- 
pling. Burbidge’s results from bus route 99 would generate questionable data, but he seemed to realize 
the fallacy of many of his assumptions once he spoke with bus route 99’s driver—something he should 
have done during exploration prior to designing the sampling plan. 

While probability sampling may be superior in theory, there are breakdowns in its application. Even 
carefully stated random sampling procedures may be subject to careless application by the people 
involved. Thus, the ideal probability sampling may be only partially achieved because of the human 
element. 

It is also possible that nonprobability sampling may be the only feasible alternative. The total popu- 
lation may not be available for study in certain cases. At the scene of a major event, it may be infeasible 
to attempt to construct a probability sample. A study of past correspondence between two companies 
must use an arbitrary sample because the full correspondence is normally not available. 

In another sense, those who are included in a sample may select themselves. In mail surveys, those 
who respond may not represent a true cross section of those who receive the questionnaire. The receivers 
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of the questionnaire decide for themselves whether they will participate. In Internet surveys those who 
volunteer don’t always represent the appropriate cross section—that’s why screening questions are 
used before admitting a participant to the sample. There is, however, some of this self-selection in 
almost all surveys because every respondent chooses whether to be interviewed. 


Methods 


Convenience 


Nonprobability samples that are unrestricted are called convenience samples. They are the least reli- 
able design but normally the cheapest and easiest to conduct. Researchers or field workers have the 
freedom to choose whomever they find: thus, the name “convenience.” Examples include informal 
pools of friends and neighbors, people responding to a newspaper’s invitation for readers to state their 
positions on some public issue, a TV reporter’s “person-on-the-street” intercept interviews, or the use 
of employees to evaluate the taste of a new snack food. 

Although a convenience sample has no controls to ensure precision, it may still be a useful pro- 
cedure. Often you will take such a sample to test ideas or even to gain ideas about a subject of 
interest. In the early stages of exploratory research, when you are seeking guidance, you might use 
this approach. The results may present evidence that is so overwhelming that a more sophisticated 
sampling procedure is unnecessary. In an interview with students concerning some issue of campus 
concern, you might talk to 25 students selected sequentially. You might discover that the responses are 
so overwhelmingly one-sided that there is no incentive to interview further. 


Purposive Sampling 


A nonprobability sample that conforms to certain criteria is called purposive sampling. There are two 
major types—judgment sampling and quota sampling. 

Judgment sampling occurs when a researcher selects sample members to conform to some crite- 
rion. In a study of labor problems, you may want to talk only with those who have experienced on-the- 
job discrimination. Another example of judgment sampling occurs when election results are predicted 
from only a few selected precincts that have been chosen because of their predictive record in past 
elections. Burbidge chose bus route 99 because the current route between East City and West City led 
him to believe that he could get a representation of both East City and West City riders. 

When used in the early stages of an exploratory study, a judgment sample is appropriate. When 
one wishes to select a biased group for screening purposes, this sampling method is also a good 
choice. Companies often try out new product ideas on their employees. The rationale is that one 
would expect the firm’s employees to be more favorably disposed toward a new product idea than 
the public. If the product does not pass this group, it does not have prospects for success in the 
general market. 

Quota sampling is the second type of purposive sampling. We use it to improve representativeness. 
The logic behind quota sampling is that certain relevant characteristics describe the dimensions of the 
population. If a sample has the same distribution on these characteristics, then it is likely to be repre- 
sentative of the population regarding other variables on which we have no control. Suppose the student 
body of Metro U is 55 percent female and 45 percent male. The sampling quota would call for sam- 
pling students at a 55 to 45 percent ratio. This would eliminate distortions due to a nonrepresentative 
gender ratio. Burbidge could have improved his nonprobability sampling by considering time-of-day 
and day-of-week variations and choosing to distribute surveys to bus route 99 riders at various times, 
thus creating a quota sample. 

In most quota samples, researchers specify more than one control dimension. Each should meet 
two tests: it should (1) have a distribution in the population that we can estimate and (2) be pertinent 
to the topic studied. We may believe that responses to a question should vary depending on the gender 
of the respondent. If so, we should seek proportional responses from both men and women. We may 
also feel that undergraduates differ from graduate students, so this would be a dimension. Other dimen- 
sions, such as the student’s academic discipline, ethnic group, religious affiliation, and social group 
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affiliation, also may be chosen. Only a few of these controls can be used. To illustrate, suppose we 
consider the following: 


Gender: Two categories—male, female. 

Class level: Two categories—graduate, undergraduate. 

College: Six categories—arts and science, agriculture, architecture, business, engineering, other. 
Religion: Four categories—Protestant, Catholic, Jewish, other. 

Fraternal affiliation: Two categories—member, nonmember. 


Family social-economic class: Three categories—upper, middle, lower. 


In an extreme case, we might ask an interviewer to find a male undergraduate business student who 
is Catholic, a fraternity member, and from an upper-class home. All combinations of these six factors 
would call for 288 such cells to consider. This type of control is known as precision control. It gives 
greater assurance that a sample will be representative of the population. However, it is costly and too 
difficult to carry out with more than three variables. 

When we wish to use more than three control dimensions, we should depend on frequency control. 
With this form of control, the overall percentage of those with each characteristic in the sample should 
match the percentage holding the same characteristic in the population. No attempt is made to find a 
combination of specific characteristics in a single person. In frequency control, we would probably find 
that the following sample array is an adequate reflection of the population: 


Population Sample 


ale 65% 67% 


Undergraduate 70 12 


Independent 15 13 


Quota sampling has several weaknesses. First, the idea that quotas on some variables assume a rep- 
resentativeness on others is argument by analogy. It gives no assurance that the sample is representative 
of the variables being studied. Often, the data used to provide controls might be outdated or inaccurate. 
There is also a practical limit on the number of simultaneous controls that can be applied to ensure pre- 
cision. Finally, the choice of subjects is left to field workers to make on a judgmental basis. They may 
choose only friendly looking people, people who are convenient to them, and so forth. 

Despite the problems with quota sampling, it is widely used by opinion pollsters and marketing and 
business researchers. Probability sampling is usually much more costly and time-consuming. Advo- 
cates of quota sampling argue that although there is some danger of systematic bias, the risks are usu- 
ally not that great. Where predictive validity has been checked (e.g., in election polls), quota sampling 
has been generally satisfactory. 


Snowball 


This design has found a niche in recent years in applications where respondents are difficult to identify 
and are best located through referral networks. It is also especially appropriate for some qualitative stud- 
ies. In the initial stage of snowball sampling, individuals are discovered and may or may not be selected 
through probability methods. This group is then used to refer the researcher to others who possess similar 
characteristics and who, in turn, identify others. Similar to a reverse search for bibliographic sources, the 
“snowball” gathers subjects as it rolls along. Various techniques are available for selecting a nonprobability 
snowball with provisions for error identification and statistical testing. Let’s consider a brief example. 
The high end of the U.S. audio market is composed of several small firms that produce ultra- 
expensive components used in recording and playback of live performances. A risky new technology 
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for improving digital signal processing is being contemplated by one firm. Through its contacts with a 
select group of recording engineers and electronics designers, the first-stage sample may be identified 
for interviewing. Subsequent interviewees are likely to reveal critical information for product develop- 
ment and marketing. 

Variations on snowball sampling have been used to study drug cultures, teenage gang activities, 
power elites, community relations, insider trading, and other applications where respondents are 
difficult to identify and contact. 


ee 


1 Sampling is based on two premises. One is that there is simplest type of probability approach is simple random 


enough similarity among the elements in a population that 

a few of these elements will adequately represent the char- 
acteristics of the total population. The second premise is 
that although some elements in a sample underestimate a 
population value, others overestimate this value. The result 
of these tendencies is that a sample statistic such as the 
arithmetic mean is generally a good estimate of a population 
mean. 


2 Agood sample has both accuracy and precision. An 


accurate sample is one in which there is little or no bias or 
systematic variance. A sample with adequate precision is 
one that has a sampling error that is within acceptable limits 
for the study’s purpose. 


3 In developing a sample, five procedural questions need to 


be answered: 

a What is the target population? 

b What are the parameters of interest? 

c What is the sampling frame? 

d What is the appropriate sampling method? 
e What size sample is needed? 


4 Avariety of sampling techniques are available. They may be 


classified by their representation basis and element selec- 
tion techniques. 


Representation Basis 


Element Selection Probability Nonprobability 


Unrestricted Simple random Convenience 


Probability sampling is based on random selection—a 
controlled procedure that ensures that each population ele- 
ment is given a known nonzero chance of selection. The 


sampling. In this design, each member of the population 
has an equal chance of being included in a sample. In con- 
trast, nonprobability selection is “not random.” When each 
sample element is drawn individually from the population at 
large, this is unrestricted sampling. Restricted sampling cov- 
ers those forms of sampling in which the selection process 
follows more complex rules. 


Complex sampling is used when conditions make simple 
random samples impractical or uneconomical. The four 
major types of complex random sampling discussed in this 
chapter are systematic, stratified, cluster, and double sam- 
pling. Systematic sampling involves the selection of every 
kth element in the population, beginning with a random start 
between elements from 1 to k. Its simplicity in certain cases 
is its greatest value. 

Stratified sampling is based on dividing a population into 
subpopulations and then randomly sampling from each of 
these strata. This method usually results in a smaller total 
sample size than would a simple random design. Stratified 
samples may be proportionate or disproportionate. 

In cluster sampling, we divide the population into con- 
venient groups and then randomly choose the groups to 
study. It is typically less efficient from a statistical viewpoint 
than the simple random because of the high degree of 
homogeneity within the clusters. Its great advantage is its 
savings in cost—if the population is dispersed geographi- 
cally—or in time. The most widely used form of clustering is 
area sampling, in which geographic areas are the selection 
elements. 

At times it may be more convenient or economical to 
collect some information by sample and then use it as a 
basis for selecting a subsample for further study. This pro- 
cedure is called double sampling. 

Nonprobability sampling also has some compelling prac- 
tical advantages that account for its widespread use. Often 
probability sampling is not feasible because the popula- 
tion is not available. Then, too, frequent breakdowns in the 
application of probability sampling discount its technical 
advantages. You may find also that a true cross section is 
often not the aim of the researcher. Here the goal may be 
the discovery of the range or extent of conditions. Finally, 
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nonprobability sampling is usually less expensive to conduct 


than is probability sampling. 


Convenience samples are the simplest and least reliable 
forms of nonprobability sampling. Their primary virtue is low 


cost. One purposive sample is the judgmental sample, in 


subjects. The other purposive sample is the quota sample. 
Subjects are selected to conform to certain predesignated 
control measures that secure a representative cross section 
of the population. Snowball sampling uses a referral ap- 
proach to reach particularly hard-to-find respondents. 


which one is interested in studying only selected types of 
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area sampling 356 population 338 sampling 338 


census 338 population element 338 sampling error 341 


cluster sampling 355 population parameters 345 sequential sampling 357 


convenience samples 359 population proportion of 


incidence 346 


simple random sample 349 


disproportionate stratified 
sampling 354 


skip interval 350 


probability sampling 343 snowball sampling 360 


double sampling 357 proportionate stratified sampling 354 


stratified random sampling 351 


judgment sampling 359 quota sampling 359 


systematic sampling 350 


multiphase sampling 357 sample frame 338 


systematic variance 340 
nonprobability sampling 343 


—_ | 


Terms in Review 
1 Distinguish between: 


sample statistics 345 


Making Research Decisions 
6 Your task is to interview a representative sample of 


Statistic and parameter. 


Sample frame and population. 


a 
b 
c Restricted and unrestricted sampling. 
d Simple random and complex random sampling. 
e Convenience and purposive sampling. 
f Sample precision and sample accuracy. 
g Systematic and error variance. 
h Variable and attribute parameters. 
i Proportionate and disproportionate samples. 
2 Under what kind of conditions would you recommend: 
a Aprobability sample? a nonprobability sample? 


b Asimple random sample? a cluster sample? a stratified 
sample? 


ce Adisproportionate stratified probability sample? 


3 You plan to conduct a survey using unrestricted sampling. 
What subjective decisions must you make? 


4 Describe the differences between a probability sample and 
a nonprobability sample. 


5 Why would a researcher use a quota purposive sample? 


attendees for the large concert venue where you work. The 
new season schedule includes 200 live concerts featuring 
all types of musicians and musical groups. Since neither 
the number of attendees nor their descriptive characteris- 
tics are Known in advance, you decide on nonprobability 
sampling. Based on past seating configurations, you 

can calculate the number of tickets that will be available 
for each of the 200 concerts. Thus, collectively, you will 
know the number of possible attendees for each type of 
music. From attendance research conducted at concerts 
held by the Glacier Symphony during the previous two 
years, you can obtain gender data on attendees by type 
of music. How would you conduct a reasonably reliable 
nonprobability sample? 

Your large firm is about to change to a customer-centered 
organization structure, in which employees who have rarely 
had customer contact will now likely significantly influence 
customer satisfaction and retention. As part of the transition, 
your superior wants an accurate evaluation of the morale of 
the firm’s large number of computer technicians. What type 
of sample would you draw if it was to be an unrestricted 
sample? 
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Bringing Research to Life From the Headlines 
8 Design an alternative nonprobability sample that will be 11. When Nike introduced its glow-in-the-dark Foamposite 
more representative of infrequent and potential riders for One Galaxy sneakers, fanatics lined up at distributors 
the CityBus project. around the country. As crowds became restless, jockey- 
9 How would you draw a cluster sample for the CityBus ing for position at the front of increasingly long lines for the 
project? limited-supply shoes, Footlocker cancelled some events. 


It’s been suggested that Nike should sell its limited-release 
introductions online rather than in stores to avoid putting 
its customer’s safety in jeopardy. What sample group 
would you suggest Nike use to assess this suggestion? 


From Concept to Practice 

10 Using Exhibit 14-8 as your guide, for each sampling tech- 
nique describe the sample frame for a study of employers’ 
skill needs in new hires using the industry in which you are 
currently working or wish to work. 


* 
FC 
ee Akron Children’s Hospital & Ohio Lottery: Innovative Research Design Drives 
Winning 
Calling Up Attendance 
ee Pebble Beach Co. 
& Campbell-Ewald Pumps Awareness 
into the American Heart Association ee Starbucks, Bank One, and Visa Launch Starbucks 


Card Duetto Visa 


Campbell-Ewald: R-E-S-P-E-C-T 


Spells Loyalty State Farm: Dangerous Intersections 
Can Research Rescue the Red The Catalyst for Women in Financial Services 
Cross? 


USTA: Come Out Swinging 
& Goodyear’s Aquatred 


Volkswagen’s Beetle 


Inquiring Minds Want to 
Know—NOW! 


Marcus Thomas LLC Tests Hypothesis 
for Troy-Bilt Creative Development 


* You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 


Determining Sample Size 


Basic Concepts for Sampling 


In the Metro University Dining Club study, we explore 
probability sampling and the various concepts used to de- 
sign the sampling process. 

Exhibit 14a-1 shows the Metro U dining club study pop- 
ulation (V = 20,000) consisting of five subgroups based on 
their preferred lunch times. The values | through 5 rep- 
resent the preferred lunch times of 11 a.m., 11:30 a.m., 
12 noon, 12:30 p.m., and | p.m. The frequency of re- 
sponse (f) in the population distribution, shown beside the 
population subgroup, is what would be found if a census 
of the elements was taken. Normally, population data are 


unavailable or are too costly to obtain. We are pretending 
omniscience for the sake of the example. 


Point Estimates 


Now assume we sample 10 elements from this population 
without knowledge of the population’s characteristics. We 
use a sampling procedure from a statistical software pro- 
gram, a random number generator, or a table of random 
numbers. Our first sample (n, = 10) provides us with the 
frequencies shown below sample n, in Exhibit 14a-1. We 
also calculate a mean score, X fee 3.0, for this sample. This 
mean would place the average preferred lunch time at 12 
noon. The mean is a point estimate and our best predictor 
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Population 
of preferred 
lunch times 


Samples 
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Y=Time f 


11:00 am. 2,000 
11:30 am. 4,000 
12:00 p.m. 7,000 
12:30 p.m. 4,000 
1:00 p.m. 3,000 
‘N= 20,000 
bt = 3.1 or 12:03 p.m. 
6 = 0.74 or 22.2 minutes 


f 
0 
1 
5 
1 
3 


= 
oO 


of the unknown population mean, p (the arithmetic aver- 
age of the population). Assume further that we return the 
first sample to the population and draw a second, third, 
and fourth sample by the same procedure. The frequen- 
cies, means, and standard deviations are as shown in the 
exhibit. As the data suggest, each sample shares some sim- 
ilarities with the population, but none is a perfect duplica- 
tion because no sample perfectly replicates its population. 


Interval Estimates 


We cannot judge which estimate is the true mean (ac- 
curately reflects the population mean). However, we can 
estimate the interval in which the true u will fall by using 
any of the samples. This is accomplished by using a for- 
mula that computes the standard error of the mean. 

o 


OY =TR 
where 
o; = standard error of the mean or the standard devia- 
tion of all possible X s 


o = population standard deviation 


n = sample size 


The standard error of the mean measures the standard 
deviation of the distribution of sample means. It varies 
directly with the standard deviation of the population from 
which it is drawn (see Exhibit 14a-2): If the standard de- 
viation is reduced by 50 percent, the standard error will 
also be reduced by 50 percent. It also varies inversely with 
the square root of the sample size. If the square root of the 
sample size is doubled, the standard error is cut by one- 
half, provided the standard deviation remains constant. 

Let’s now examine what happens when we apply sam- 
ple data (1,) from Exhibit 14a-1 to the formula. The sample 
standard deviation from sample n will be used as an unbi- 
ased estimator of the population standard deviation. 


On 2 
where 
s = standard deviation of the sample, n, 
n, = 10 
X, = 3.0 
gs, = 1.15 


Substituting into the equation: 


2.64 


3.00 
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Estimating the Population Mean 


How does this improve our prediction of u from X? The 
standard error creates the interval range that brackets the 
point estimate. In this example, j is predicted to be 3.0 or 
12 noon (the mean of n,) £0.36. This range may be visual- 
ized on a continuum (see diagram at bottom of page). 

We would expect to find the true w between 2.64 
and 3.36—between 11:49 a.m. and 12:11 p.m. Gif 2 = 
11:30 a.m. and 0.64 = (30 minutes) = 19.2 minutes, then 
2.64 = 11:30 a.m. + 19.2 minutes, or 11:49 a.m.). Since 
we assume omniscience for this illustration, we know the 
population average value is 3.1. Further, because standard 
errors have characteristics like other standard scores, we 
have 68 percent confidence in this estimate—that is, one 
standard error encompasses +1 Z or 68 percent of the 
area under the normal curve (see Exhibit 14a-3). Recall 
that the area under the curve also represents the confi- 
dence estimates that we make about our results. The com- 
bination of the interval range and the degree of confidence 
creates the confidence interval. To improve confidence to 
95 percent, multiply the standard error of 0.36 by + 1.96 
(Z), since 1.96 Z covers 95 percent of the area under the 
curve (see Exhibit 14a-4). Now, with 95 percent confi- 
dence, the interval in which we would find the true mean 
increases to + 0.70 (from 2.3 to 3.7 or from 11:39 a.m. to 
12:21 p.m.). 

Parenthetically, if we compute the standard deviation 
of the distribution of sample means in Exhibit 14a-1, [3.0, 
2.8, 3.6, 3.3], we will discover it to be 0.35. Compare this 
to the standard error from the original calculation (0.36). 
The result is consistent with the second definition of the 
standard error: the standard deviation of the distribution of 
sample means (n,, N,, N3, and n ne Now let’s return to the 
dining club example and apply some of these concepts to 
the researchers’ problem. 

If the researchers were to interview all the students and 
employees in the defined population, asking them, How 
many times per month would you eat at the club? they 
would get a distribution something like that shown in part 
A of Exhibit 14a-5. The responses would range from zero 
to as many as 30 lunches per month with a p and o. 

However, they cannot take a census, so p and o remain 
unknown. By sampling, the researchers find the mean to be 
10.0 and the standard deviation to be 4.1 eating experiences 
(how often they would eat at the club per month). In part 
C of Exhibit 14a-5, three observations about this sample 
distribution are consistent with our earlier illustration. First, 


True mean = 3.1 


3.36 


11:49 a.m. 


>< 


ABIL osai. 
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>Exhibit 14a-2 Effects on Standard Error of Mean of Increasing Precision 


Reducing the Standard 


Deviation by 50% Quadrupling the Sample 
2 eo oad acer 
on =—= z= =0. on = =0. 
Vn V10 V25 
0.37 0.8 
Om = SS = O17 ox =— = = 0.08 
V 10 100 
where 


ox = standard error of the mean 
ox = standard deviation of the sample 
n = sample size 


Note: A 400 percent increase in sample size (from 25 to 100) would yield only a 200 percent 
increase in precision (from 0.16 to 0.08). Researchers are often asked to increase precision, but the 
question should be, at what cost? Each of those additional sample elements adds both time and 
cost to the study. 


>Exhibit 14a-3 Confidence Levels and the Normal Curve 


95% 


68% 


—1.960x% —-lox 0 +10x% +1.960x 


>Exhibit 14a-4 Standard Errors Associated with Areas under the Normal Curve 


Standard Error (Z) Percent of Area* Approximate Degree of Confidence 
1.00 68.27 68% 
1.65 90.10 90 
1.96 95.00 95 
3.00 SONS) eg) 


*Includes both tails in a normal distribution. 


it is shown as a histogram; it represents a frequency distri- If the researchers could draw repeated samples as we 
bution of empirical data, while the smooth curve of part A did earlier, they could plot the mean of each sample to se- 
is a theoretical distribution. Second, the sample distribution cure the solid line distribution found in part B. According 
(part C) is similar in appearance but is not a perfect duplica- to the central limit theorem, for sufficiently large samples 
tion of the population distribution (part A). Third, the mean (n = 30), the sample means will be distributed around the 
of the sample differs from the mean of the population. population mean approximately in a normal distribution. 
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>Exhibit 14a-5 A Comparison of Population Distribution, Sample Distribution, and 
Distribution of Sample Means of Metro U Dining Club Study 


Population distribution 


Distribution of means 
from repeated samples 
of a fixed size (n = 64) 


fa 68% of the area 


Sample distribution 


|< $<. S$ $$ | <_< § ———— > | <_$§__ § — > |< — Ss ——— 


10.0 
x 


Note: The distributions in these figures are not to scale, but this fact is not critical to our 


understanding of the dispersion relationship depicted. 


Even if the population is not normally distributed, the dis- 
tribution of sample means will be normal if there is a large 
enough set of samples. 


Estimating the Interval for the Metro U Dining 
Club Sample 


Any sample mean will fall within the range of the distribu- 
tion extremes shown in part B of Exhibit 14a-5. We also 
know that about 68 percent of the sample means in this 
distribution will fall between x, and x, and 95 percent will 
fall between x, and x,. 


If we project points x, and x, up to the population dis- 
tribution (part A of Exhibit 14a-5) at points x’, and x’,, 
we see the interval where any given mean of a random 
sample of 64 is likely to fall 95 percent of the time. Since 
we will not know the population mean from which to 
measure the standard error, we infer that there is also a 
95 percent chance that the population mean is within two 
standard errors of the sample mean (10.0). This inference 
enables us to find the sample mean, mark off an interval 
around it, and state a confidence likelihood that the popu- 
lation mean is within this bracket. 
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Because the researchers are considering an investment 
in this project, they would want some assurance that the 
population mean is close to the figure reported in any sam- 
ple they take. To find out how close the population mean 
is to the sample mean, they must calculate the standard 
error of the mean and estimate an interval range within 
which the population mean is likely to be. 

Given a sample size of 64, they still need a value for the 
standard error. Almost never will one have the value for 
the standard deviation of the population (oc), so we must 
use a proxy figure. The best proxy for o is the standard 
deviation of the sample (s). Here the standard deviation 
(s = 4.1) was obtained from a pilot sample: 


—~ s _ 41 _ 


If one standard error of the mean is equal to 0.51 visit, then 
1.96 standard errors (95 percent) are equal to 1.0 visit. The 
students can estimate with 95 percent confidence that the 
population mean of expected visits is within 10.0 + 1.0 
visit, or from 9.0 to 11.0 mean visits per month. We dis- 
cuss pilot tests as part of the pretest phase in Chapter 13. 


Changing Confidence Intervals 


The preceding estimate may not be satisfactory in two 
ways. First, it may not represent the degree of confidence 
the researchers want in the interval estimate, considering 
their financial risk. They might want a higher degree of 
confidence than the 95 percent level used here. By re- 
ferring to a table of areas under the normal curve, they 
can find various other combinations of probability. Ex- 
hibit 14a-6 summarizes some of those more commonly 
used. Thus, if the students want a greater confidence in 
the probability of including the population mean in the 
interval range, they can move to a higher standard error, 
say, X + 30. Now the population mean lies somewhere 
between 10.0 + 3 (0.51) or from 8.47 to 11.53. With 
99.73 percent confidence, we can say this interval will in- 
clude the population mean. 

We might wish to have an estimate that will hold for a 
much smaller range, for example, 10.0 + 0.2. To secure 


this smaller interval range, we must either (1) accept a 
lower level of confidence in the results or (2) take a sam- 
ple large enough to provide this smaller interval with the 
higher desired confidence level. 

If one standard error is equal to 0.51 visit, then 0.2 visit 
would be equal to 0.39 standard error (0.2/0.51 = 0.39). 
Referring to a table of areas under the normal curve (Ap- 
pendix C, Exhibit C-1), we find that there is a 30.3 percent 
chance that the true population mean lies within £0.39 
standard error of 10.0. With a sample of 64, the sample 
mean would be subject to so much error variance that only 
30 percent of the time could the researchers expect to find 
the population mean between 9.8 and 10.2. This is such 
a low level of confidence that the researchers would nor- 
mally move to the second alternative; they would increase 
the sample size until they could secure the desired interval 
estimate and degree of confidence. 


Calculating the Sample Size for 
Questions Involving Means 


Before we compute the desired sample size for the Metro 
U dining club study, let’s review the information we will 
need: 


1. The precision desired and how to quantify it: 


a. The confidence level we want with our estimate. 
b. The size of the interval estimate. 


2. The expected dispersion in the population for the 
investigative question used. 


3. Whether a finite population adjustment is needed. 


The researchers have selected two investigative ques- 
tion constructs as critical—‘“‘frequency of patronage” and 
“interest in joining’—because they believe both to be cru- 
cial to making the correct decision on the Metro U dining 
club opportunity. The first requires a point estimate, the 
second a proportion. By way of review, decisions needed 
and decisions made by Metro U researchers are summa- 
rized in Exhibit 14a-7. 


>Exhibit 14a-6 Estimates Associated with Various Confidence Levels in the Metro U Dining 


Club Study 


Approximate Degree of Confidence 


68% 
90% 
95% 
99% 


Interval Range of Dining Visits per Month 
u is between 9.48 and 10.52 visits 
u is between 9.14 and 10.86 visits 
u is between 8.98 and 11.02 visits 


u is between 8.44 and 11.56 visits 


>chapter 14 Sampling 369 


>Exhibit 14a-7 Metro U Sampling Design Decision on “Meal Frequency” and “Joining” Constructs 


Sampling Issues 


Metro U Decisions 


“Meal Frequency” 
(interval, ratio data) 


“Joining” 
(nominal, ordinal data) 


1. The precision desired and how to quantify it: 


e The confidence researcher wants in the estimate 
(selected based on risk) 


e The size of the interval estimate the researcher will 
accept (based on risk) 


95% confidence (Z = 1.96) 


+0.5 meal per month 


95% confidence (Z = 1.96) 


+0.10 (10 percent) 


Measure of Dispersion 


e Standard deviation 


e Measure of sample dispersion 


4. Estimate of standard deviation of population: 
e Standard error of mean 


e Standard error of the proportion 


0.5/1.96 = 0.255 


pq = 0.30(0.70) = 0.21 


0.10/1.96 = 0.051 


6. Calculated sample size 


n = 259* 


i =H 


*Because both investigative questions were of interest, the researcher would use the larger of the two sample sizes calculated, 


n = 259 for the study. 


Precision 


With reference to precision, the 95 percent confidence 
level is often used, but more or less confidence may be 
needed in light of the risks of any given project. Similarly, 
the size of the interval estimate for predicting the popula- 
tion parameter from the sample data should be decided. 
When a smaller interval is selected, the researcher is say- 
ing that precision is vital, largely because inherent risks 
are high. For example, on a 5-point measurement scale, 
one-tenth of a point is a very high degree of precision in 
comparison to a |-point interval. Given that a patron could 
eat up to 30 meals per month at the dining club (30 days 
times | meal per day), anything less than one meal per 
day would be asking for a high degree of precision in the 
Metro U study. The high risk of the Metro U study war- 
rants the 0.5 meal precision selected. 


Population Dispersion 


The next factor that affects the size of the sample for 
a given level of precision is the population dispersion. 
The smaller the possible dispersion, the smaller will 
be the sample needed to give a representative picture 
of population members. If the population’s number of 
meals ranges from 18 to 25, a smaller sample will give 
us an accurate estimate of the population’s average meal 
consumption. However, with a population dispersion 
ranging from 0 to 30 meals consumed, a larger sample 
is needed for the same degree of confidence in the esti- 
mates. Since the true population dispersion of estimated 
meals per month eaten at Metro U dining club is un- 
knowable, the standard deviation of the sample is used 
as a proxy figure. Typically, this figure is based on any 
of the following: 
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e Previous research on the topic. 


e A pilot test or pretest of the data instrument among 
a sample drawn from the population. 


e A rule of thumb (one-sixth of the range based 
on six standard deviations within 99.73 percent 
confidence). 


If the range is from 0 to 30 meals, the rule-of-thumb 
method produces a standard deviation of 5 meals. The 
researchers want more precision than the rule-of-thumb 
method provides, so they take a pilot sample of 25 and 
find the standard deviation to be 4.1 meals. 


Population Size 


A final factor affecting the size of a random sample is the 
size of the population. When the size of the sample ex- 
ceeds 5 percent of the population, the finite limits of the 
population constrain the sample size needed. A correction 
factor is available in that event. 

The sample size is computed for the first construct, 
meal frequency, as follows: 


Ss 
Ox Va 
Vn = a 
2 
n= oF 
— (4.19 
~ {02557 


n = 258.5 or 259 


where 
Ox = 0.255 (0.51/1.96) 


If the researchers are willing to accept a larger interval 
range (+1 meal), and thus a larger amount of risk, then 
they can reduce the sample size to n = 65. 


Calculating the Sample Size for 
Questions Involving Proportions 


The second key question concerning the dining club study 
was, what percentage of the population says it would join 
the dining club, based on the projected rates and services? 
In business, we often deal with proportion data. An ex- 
ample is a CNN poll that projects the percentage of people 
who expect to vote for or against a proposition or a can- 
didate. This is usually reported with a margin of error of 
+5 percent. 

In the Metro U study, a pretest answers this question 
using the same general procedure as before. But instead of 
the arithmetic mean, with proportions, it is p (the propor- 
tion of the population that has a given attribute)'—in this 


case, interest in joining the dining club. And instead of the 
standard deviation, dispersion is measured in terms of p X 
q (in which q is the proportion of the population not having 
the attribute), and g = (1 — p). The measure of dispersion 
of the sample statistic also changes from the standard error 
of the mean to the standard error of the proportion o,,. 

We calculate a sample size based on these data by mak- 
ing the same two subjective decisions—deciding on an 
acceptable interval estimate and the degree of confidence. 
Assume that from a pilot test, 30 percent of the students 
and employees say they will join the dining club. We 
decide to estimate the true proportion in the population 
within 10 percentage points of this figure (p = 0.30 + 
0.10). Assume further that we want to be 95 percent con- 
fident that the population parameter is within £0.10 of 
the sample proportion. The calculation of the sample size 
proceeds as before: 


+0.10 = desired interval range within which the 
population proportion is expected (subjective 
decision) 


1.960, = 95 percent confidence level for estimating the 
interval within which to expect the population 
proportion (subjective decision) 

o, = 0.051 = standard error of the proportion 
(0.10/1.96) 

pq = measure of sample dispersion (used here as an 
estimate of the population dispersion) 


n = sample size 


— 4/P4d 
5, — TD 
_ Pq 
n o 
_ 03X07 
(0.051)? 
n= 8l 


The sample size of 81 persons is based on an infinite 
population assumption. If the sample size is less than 
5 percent of the population, there is little to be gained by 
using a finite population adjustment. The students inter- 
preted the data found with a sample of 81 chosen randomly 
from the population as: We can be 95 percent confident 
that 30 percent of the respondents would say they would 
join the dining club with a margin of error of +10 percent. 

Previously, the researchers used pilot testing to gener- 
ate the variance estimate for the calculation. Suppose this 
is not an option. Proportions data have a feature concern- 
ing the variance that is not found with interval or ratio 
data. The pq ratio can never exceed 0.25. For example, if 
p = 0.5, then g = 0.5, and their product is 0.25. If either p 
or q is greater than 0.5, then their product is smaller than 


0.25 (0.4 X 0.6 = 0.24, and so on). When we have no in- 
formation regarding the probable p value, we can assume 
that p = 0.5 and solve for the sample size. 


— Pq 


n= 7 
— (0.50)(0.50) 
(0.51) 
_ 0.25 
(0.051)? 


n= 96 


where 
Pq = measure of dispersion 
n = sample size 
c= standard error of the proportion 
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If we use this maximum variance estimate in the dining 
club example, we find the sample size needs to be 96 per- 
sons in order to have an adequate sample for the question 
about joining the club. 

When there are several investigative questions of 
strong interest, researchers calculate the sample size for 
each such variable—as we did in the Metro U study for 
“meal frequency” and “joining.” The researcher then 
chooses the calculation that generates the largest sample. 
This ensures that all data will be collected with the neces- 
sary level of precision. 


Analysis and Presentation of Data 


chapter 15 


learningobjectives 
After reading this chapter, you should understand... 


The importance of editing the collected raw data to detect errors and omissions. 

How coding is used to assign numbers and other symbols to answers and to categorize responses. 
The use of content analysis to interpret and summarize open questions. 

Problems with and solutions for “don’t know” responses and missing data. 


The options for data entry and manipulation. 


6 6 Pattern thinking, where you look at what’s 
working for someone else and apply it 
to your own situation, is one of the best 


ways to make big things happen for you 
and your team. 9 9 


David Novak, chairman and CEO, 
Yum! Brands, Inc. 


bringingresearchtolife ...<° 


“Is my being early for our meeting a problem?” asks 
Myra as she moves past a pile of computer printouts 
stacked precariously high just inside the door to Jason’s 
office. “Might the industrious team in your outer office 
be studying my MindWriter Project 2 data?” 

“Not yet,” comments Jason as he waves Myra to an 
empty chair. “Just give me one second.” He quickly 
writes two notes on Post-its and slaps one on a pencil 
sketch of a graph and attaches the other to a histogram. 
“Sammye, you want to come get these?” Jason calls to 
one of the team members in the outer office. 

Meanwhile, Myra chooses an available chair and 
waits. She is here to convince Jason to take on yet 
another project for MindWriter. This one has a short 
turnaround. Turning his attention to Myra, Jason 
extracts a folder lying on the credenza behind him. 
“Actually those worker bees are new members of my 
staff, graduate students from the university. They’re 
assigned to the City Center for Performing Arts project,” 
shares Jason. “It’s because of your recommendation 
that we got the job. I thought you knew.” 

“Of course I knew. I’ve been serving on CCPA’s 
board for two years. Will you be presenting the 
preliminary analysis at the next meeting this Friday?” 

“As in day after tomorrow?” asks Jason. “Only 
in our dreams! The preliminary analysis you see 
them working on is strictly for us. Although we may 
develop presentation charts that might be presented to 
the Center Board, it is just as likely that none of the 
material you see stacked here will end up in the report 
as is. We are nowhere near ready to write the client 
report. We just finished cleaning the data file yesterday. 


This morning I ran a full set of frequencies. Jill, David, 
and Sammye started their preliminary analysis ... uh, 
90 minutes ago.” 

“So I guess [ll have to wait until you have 
something more solid to learn even a briefing on what 
you’ ve found so far?” inquires Myra, smiling. 

“Ah,” smiles Jason in return, “you’ ve learned H&A’s 
process fairly well.” Myra grins and then modifies her 
position in the chair, leaning slightly toward Jason. Just 
before she speaks, Jason observes, “Oh, no! You’re 
changing into your ‘It’s time to get down to business’ 
posture. So what’s the new project you want to discuss... 
and the impossible deadline you need me to meet?” 

“Just hear me out, Jason. MindWriter’s LT3000 
product group has decided it needs to use “superiority 
in custom-designed systems’ as its claim in a new ad 
campaign, but legal says we don’t have enough data to 
support the claim. The ad agency we have chosen has 
a short window of opportunity. We need supporting 
data within 10 days.” Myra holds up her hand to stop 
the objection she anticipates from Jason. “We know you 
don’t have time to collect new primary data and analyze 
itin 10 days ... so I brought the next best thing. I’ve got 
three boxes of miscellaneous records in my trunk ...” 

“Let’s go see what you brought me,” Jason groans 
good-naturedly as he unfolds himself from his chair. 
“Then we’ ll see if this project is even feasible.” As he 
passes through the outer office, Jason motions for one 
of the students to follow; then in an aside to Myra he 
said, “Myra, meet David Chesley. You're just lucky 
that my new interns are all so eager that they will enjoy 
juggling two projects at one time.” 
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> Introduction 


Once the data begin to flow, a researcher’s attention turns to data analysis. This chapter focuses on the 
first phases of that process, data preparation and description. Data preparation includes editing, cod- 
ing, and data entry and is the activity that ensures the accuracy of the data and their conversion from 
raw form to reduced and classified forms that are more appropriate for analysis. Preparing a descriptive 
statistical summary is another preliminary step leading to an understanding of the collected data. It is 
during this step that data entry errors may be revealed and corrected. Exhibit 15-1 reflects the steps in 
this phase of the research process. 


>Exhibit 15-1 Data Preparation in the Research Process 


Instrument Design 


Final Pretested Precode the 
Instrument Instrument 
Coded & Pretested 
Instrument 


| Data Collection & Preparation 
Collect Data 

Postcode the erieribee Postcode Free- 

Instrument AEN) Response Questions 


Edit Data File 
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> Editing 


The customary first step in analysis is to edit the raw data. Editing detects errors and omissions, cor- 
rects them when possible, and certifies that maximum data quality standards are achieved. The editor’s 
purpose is to guarantee that data are: 

° Accurate. 

¢ Consistent with the intent of the question and other information in the survey. 

¢ Uniformly entered. 

¢ Complete. 

e Arranged to simplify coding and tabulation. 


In the following question asked of adults 18 or older, one respondent checked two categories, indi- 
cating that he was a retired officer and currently serving on active duty. 


Please indicate your current military status: 


Y] Active duty Reserve 7] Retired 


LC National Guard L] Separated L) Never served in the military 


The editor’s responsibility is to decide which of the responses is both consistent with the intent of the 
question or other information in the survey and most accurate for this individual participant. 


Field Editing 


In large projects, field editing review is a responsibility of the field supervisor. It, too, should be done 
soon after the data have been gathered. During the stress of data collection in a personal interview and 
paper-and-pencil recording in an observation, the researcher often uses ad hoc abbreviations and spe- 
cial symbols. Soon after the interview, experiment, or observation, the investigator should review the 
reporting forms. It is difficult to complete what was abbreviated or written in shorthand or noted illeg- 
ibly if the entry is not caught that day. When entry gaps are present from interviews, a callback should 
be made rather than guessing what the respondent “probably would have said.” Self-interviewing has 
no place in quality research. 

A second important control function of the field supervisor is to validate the field results. This nor- 
mally means he or she will reinterview some percentage of the respondents, at least on some questions, 
verifying that they have participated and that the interviewer performed adequately. Many research 
firms will recontact about 10 percent of the respondents in this process of data validation. 


Central Editing 


While online surveys and their subsequent data entry has eliminated some 
editing problems, regardless of the mode of collection, all data should get y) go : 
a thorough editing. For a small study, the use of a single editor produces ‘J : ~ 
maximum consistency. In large studies, editing tasks should be allocated so r) a 8 ) é 
that each editor deals with one entire section. Although the latter approach 
will not identify inconsistencies between answers in different sections, the @ 
problem can be handled by identifying questions in different sections that —_ 
might point to possible inconsistency and having one editor check the data 
generated by these questions. 

Sometimes it is obvious that an entry is incorrect—for example, when Data quality can’t be left to chance. Data is worth 
a respondent clearly specifies time in days (e.g., 13) when it was re- thing if itis not of good quality, so cleaning the 

: data is critical. If research delivers the right people 
quested in weeks (for example, you expect a number of 4 or less)—or es : 
: . i, . at the right time to generate the right data for a 

data is entered in the wrong place. When replies are inappropriate (out of 


decision, then the researcher isn’t gambling on the 
range of expected values or not related to the question asked) or missing, __ insights. 
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How Dirty Is the Dirty Data Problem? 


Dirty data, data that is “misleading, incorrect, without general- 
ized formatting, contains spelling or punctuation errors, is input 
incorrectly, or is falsified by the respondent”? is a major business 
problem. In this era of online data collection, data becomes dirty 
when duplicate data is created, like when a respondent “acci- 
dentally double clicks the ‘submit’ button on forms [or surveys], 
or [in a web click-through study] revisits a page which has ‘ex- 
pired’ and refreshes it.”’ Data becomes dirty when it is outdated, 
like when a prospective customer moves or changes his or her 
phone number or e-mail address. Dirty data can also result when 
there is a “disconnect between data in computer systems and 
data embedded in paper or non-machine-readable electronic 
documents.”° 

Where might dirty data be a problem? Try your doctor’s of- 
fice when she is trying to diagnose your illness or condition. Or 
your favorite restaurant is trying to decide what item to drop from 
its menu. Or a manufacturer is trying to identify machine main- 
tenance practices creating faulty parts. Or a human relations 
manager trying to distinguish which training regimen is having 
the most positive impact on the organization’s bottom line. Craig 
Focardi, writing in Mortgage Banking, describes lenders as sub- 
sisting on a “diet of missing, incomplete, and inaccurate data 
elements for loan origination.” He likens it to a “water-bucket 
brigade, where water is lost as water buckets pass from hand 
to hand.”* He believes that poor-quality data was a contribut- 
ing cause to the current mortgage liquidity crisis in the United 
States. “During the summer of 2007, subprime mortgage fund- 
ing liquidity disappeared in part because subprime lenders and 
mortgage investors lacked the data to accurately reassess and 
reprice credit, collateral, and prepayment risks.” 

To tackle the data-quality issue it helps to categorize dirty 
data problems into four categories: invalid data, incomplete 
data, inconsistent data, and incorrect data.° Invalid data is data 
that contains entry errors, like a nonexistent postal code. Incom- 
plete data is missing data that is needed to make a decision; 
deleting data from one database due to a disk crash or losing 
the link between databases in a data warehouse can create in- 
complete data, as can having data become a battleground in an 
internal turf war—“everyone clinging to [its] own little piece of the 
data store, nobody willing to share.”' Inconsistent data is often 
seen only from the big-picture perspective; data from one data- 
base might make little or no sense when seen from the perspec- 
tive of data in a separate but connected database. This might 
be due to data protocols, like naming a corporation: IBM could 
be entered as IBM, I.B.M., or International Business Machines in 


the “supplier” field of multiple databases, preventing those da- 
tabases from being correctly merged or preventing a researcher 
from discovering patterns relating to each supplier. /ncorrect 
data is just wrong; it most often occurs when data is lost, or 
falsified, or when data, like a customer order, is not entered at all. 

The basic steps for dealing with bad data are the same: de- 
tecting it and removing or correcting it. But some experts sug- 
gest a third step: determining how much error to tolerate.9 If the 
bad data exists in critical areas, like patient blood test results or 
a firm’s accounts receivables, it is likely worth the organization’s 
time and effort to correct it. If the dirty data is in the notes section 
of a salesman’s contact management file, it might not warrant 
such time and effort. The problem has gotten so bad that some 
companies have hired data stewards—a person charged with 
keeping it’s company’s data clean." 

According to research and advisory firm Gartner Inc., 25 per- 
cent of critical data in the world’s top 1000 companies is and 
will continue to be flawed.’ Other estimates suggest that num- 
ber could be as high as 30 percent! Dun & Bradstreet Corp., 
which helps companies clean their supplier files, estimates most 
companies show 20 percent duplicate supplier records.* A Price- 
waterhouseCoopers study indicated that poor data manage- 
ment cost global businesses “more than $1.4 billion per year 
in billing, accounting and inventory snafus.”' The distinguished 
British Computer Society's Roger Needham Award-winning 
professor, Wenfei Fan, estimates that “dirty data costs U.S. busi- 
nesses as much as $611 billion—and U.S. customers as much 
as $2.5 billion a year.”™ 

Most experts agree that a first step is to establish data pro- 
tocols, simplistic rules for how data is entered, like what items 
(pants, slacks, or trousers) are called or how dates are entered 
(March 20, 2010 or 20 March 10). Other suggestions include 
backing up data regularly, controlling access to data via security 
mechanisms, designing user interfaces that prevent the input of 
invalid data, and using error detection and correction software 
when transmitting data." The president of Tigris Consulting adds, 
“you need to be very careful when pulling data from different sys- 
tems together to make sure that it’s the right field name, the right 
format and that semantically, [the data] means what you want 
it to mean.”° But most data experts believe the situation won’t 
improve until top management makes clean data a priority. 
www.tigris.com; www.gartner.com; www.dnb.com; 
www.pwc.com; www.bcs.org; www.mortgagebankingmaga- 
zine.com 
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Visitors to Las Vegas have an opportunity to determine the direc- 
tion of CBS programming by visiting the CBS Television City Re- 
search Center in the MGM Grand Hotel and Casino. What makes 
Las Vegas an ideal location for a research lab is the cross sec- 
tion of the American population and the large number of interna- 
tional citizens among its 39 million visitors each year. In a typical 
screening with 250 people, individuals represent more than 
40 states and every conceivable lifestyle. Participants watch 
a 30- to 45-minute segment of a new or proposed program, 
without commercial interruptions, followed by a survey process 
lasting 15 minutes. Each seat is equipped with a computer touch 
screen linked to Nielsen Entertainment’s proprietary Nielsen 
ReelResearch. Participants share feedback on the show and 
personal demographics in real time, while network executives 


observe participants and their feedback from remote offices 
around the country. The facility, designed by GES, also provides 
focus group capabilities, used for follow-up interviews along with 
other research initiatives. Participants are compensated with a 
chance to win a home theater system, as well as a $10 coupon 
that they may redeem on program-logo T-shirts, caps, pins, key 
chains, and even computer software in the CBS Television City Re- 
search Center store. Arising out of a temporary test conducted in 
1991, the research facility today operates 12 hours per day year- 
round. So on your next visit to the City of Lights, when the slots or 
big-name entertainers lose appeal, entertain yourself with research. 

What are some of the advantages to having touch-screen 
data entry? 


www.nielsen.com; www.viad.com 


the editor can sometimes detect the proper answer by reviewing the other information in the data set. 
This practice, however, should be limited to the few cases where it is obvious what the correct answer 
is. It may be better to contact the respondent for correct information, if time and budget allow. Another 
alternative is for the editor to strike out the answer if it is inappropriate. Here an editing entry of “no 
answer” or “unknown” is called for. 

Another problem that editing can detect when using personal or phone interviews concerns faking 
an interview that never took place. This “armchair interviewing” is difficult to spot, but the editor is in 
the best position to do so. One approach is to check responses to open-ended questions. These are most 
difficult to fake. Distinctive response patterns in other questions will often emerge if data falsification 
is occurring. To uncover this, the editor must analyze as a set the instruments used by each interviewer. 

Here are some useful rules to guide editors in their work: 


¢ Be familiar with instructions given to interviewers and coders. 

¢ Do not destroy, erase, or make illegible the original entry by the interviewer or respondent; 
original entries should remain legible. 

e Make all editing entries on an instrument or in a data set in some distinctive color and in a 
standardized form. 

¢ Initial all answers changed or supplied. 


¢ Place initials and date of editing on each instrument completed or in a separate field within a 
data set. 


> Coding 


Coding involves assigning numbers or other symbols to answers so that the responses can be grouped 
into a limited number of categories. In coding, categories are the partitions of a data set of a given 
variable (e.g., if the variable is gender, the partitions are male and female). Categorization is the 
process of using rules to partition a body of data. Both closed- and open-response questions must 
be coded. Some paper and online survey questions contain numerical or textual codes and these are 
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visible to the respondent. Other questions, for example open response questions, are coded after the 
data are collected. Recoding of variables once data are initially analyzed is possible when trying to 
understand the patterns within data. 

The categorization of data sacrifices some data detail but is necessary for efficient analysis. Most 
statistical and banner/table software programs work more efficiently in the numeric mode. Instead 
of entering the word male or female in response to a question that asks for the identification of one’s 
gender, we would use numeric codes (e.g., 0 for male and | for female). Numeric coding simplifies 
the researcher’s task in converting a nominal variable, like gender, to a “dummy variable,” a topic we 
discuss in the Online Learning Center supplement: Multivariate Analysis, An Overview. Statistical 
software also can use alphanumeric codes, as when we use M and F, or other letters, in combination 
with numbers and symbols for gender. 


Codebook Construction 


A codebook, or coding scheme, contains each variable in the study and specifies the application of 
coding rules to the variable. It is used by the researcher or research staff to promote more accurate and 
more efficient data entry or data analysis. It is also the definitive source for locating the positions of 
variables in the data file during analysis. In many statistical programs, the coding scheme is integral 
to the data file. Most codebooks—computerized or not—contain the question number, variable name, 
location of the variable’s code on the input medium (e.g., spreadsheet or SPSS data file), descriptors for 
the response options, and whether the variable is alphabetic or numeric. An example of a paper-based 
codebook is shown in Exhibit 15-2. Pilot testing of an instrument provides sufficient information about 
the variables to prepare a codebook. A preliminary codebook used with pilot data may reveal coding 
problems that will need to be corrected before the data for the final study are collected and processed. 


Coding Closed Questions 


The responses to closed questions include scaled items for which answers can be anticipated. Closed 
questions are favored by researchers over open-ended questions for their efficiency and specificity. 
They are easier to code, record, and analyze. When codes are established in the instrument design 
phase of the research process, it is possible to precode the questionnaire during the design stage. With 
computerized survey design, and computer-assisted, computer-administered, or online collection of 


File Edit Yiew Insert Format Analyze Graphs Utilities Window Help 


The researcher here requested 
a frequency printout of all vari- 
ables when 83 cases had been 
entered. IBM® SPSS® Statistics 
presents them sequentially in 
one document. The left frame 
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included in this particular 
output file. Both variables 
Qual2 and Qual indicate 

3 missing cases. This would 
be a cautionary flag to a good 
researcher. During editing the 
researcher would want to verify 
that these are true instances 
where participants did not rate 
the quality of both objects, 
rather than data entry errors. 
SPSS Inc. was acquired by 
IBM in October, 2009. 
www.ibm.com/software/ 
analytics/spss 
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>Exhibit 15-2 Sample Codebook of Questionnaire Items 


Variable Variable 
Question Number Code Description Name 
1 Record number RECNUM 
2 Respondent number RESID 


2 digit birth year 
99 = Missing 


BIRTH 


4 = Separated 
5 = Never married 
9 = Missing 


4 6 Marital status MARITAL 
1 = Married 
2 = Widowéer) 
3 = Divorced 


6 Reason for purchase 

1 = Mentioned 

0 = Not mentioned 
8 Bought home HOME 
9 Birth of child BIRTHCHD 
10 Death of relative or friend DEATH 
14 Promoted PROMO 
12 Changed job/career CHGJOB 
13 Paid college expenses COLLEXP 
14 Acquired assets ASSETS 
15 Retired RETIRED 
16 Changed marital status CHGMAR 
ali ta Started business STARTBUS 
18 Expanded business EXPBUS 
19 Parent’s influence PARENT 
20 Contacted by agent AGENT 
21 Other OTHER 


data, precoding is necessary as the software tallies data as they are collected. Precoding is particu- 
larly helpful for manual data entry (e.g., from mail or intercept self-administered surveys) because it 
makes the intermediate step of completing a data entry coding sheet unnecessary. With a precoded 
instrument, the codes for variable categories are accessible directly from the questionnaire. A par- 
ticipant, interviewer, field supervisor, or researcher (depending on the data collection method) is able 
to assign the appropriate code on the instrument by checking, circling, or printing it in the proper 
coding location. 

Exhibit 15-3 shows questions in the sample codebook. When precoding is used, editing precedes 
data processing. Note question 4, in which the respondent may choose between five categories of mari- 
tal status and enter the number of the item best representing present status in the coding portion of the 
questionnaire. This code is later transferred to an input medium for analysis. 
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>Exhibit 15-3 Sample Questionnaire Items 


1. What is the zip code of your residence? 


3. Gender (1) Male Indicate 
(2) Female your choice ———> 
by number 


5. Do you own or rent your primary residence? 
(1) Own Indicate 
(2) Rent your choice ————> 


(3) Living quarters provided by number 


Coding Open-Ended Questions 


One of the primary reasons for using open-ended questions is that insufficient information or lack of a 
hypothesis may prohibit preparing response categories in advance. Researchers are forced to categorize 
responses after the data are collected. Other reasons for using open-ended responses include the need 
to measure sensitive or disapproved behavior, discover salience or importance, or encourage natural 
modes of expression.' Also, it may be easier and more efficient for the participant to write in a known 
short answer rather than read through a long list of options. Whatever the reason for their use, analyzing 
enormous volumes of open-ended questions slows the analysis process and increases the opportunity 
for error. The variety of answers to a single question can be staggering, hampering postcollection cat- 
egorization. Even when categories are anticipated and precoded for open-ended questions, once data 
are collected, researchers may find it useful to reassess the predetermined categories. One example is 
a 7-point scale in which the researcher offered the participant three levels of agreement, three levels of 
disagreement, and one neutral position. Once the data are collected, if these finer nuances of agreement 
do not materialize, the editor may choose to recategorize the data into three levels: one level of agree- 
ment, one level of disagreement, and one neutral position. 

Exhibit 15-3, question 6, illustrates the use of an open-ended question for which advance knowledge 
of response options was not available. The answer to “What prompted you to purchase your most recent 
life insurance policy?” was to be filled in by the participant as a short-answer essay. After preliminary 
evaluation, response categories (shown in the codebook, Exhibit 15-2) were created for that item. 
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Coding Rules 


Four rules guide the pre- and postcoding and categorization of a data set. The categories within a single 
variable should be: 


¢ Appropriate to the research problem and purpose. 
e Exhaustive. 
e Mutually exclusive. 


e Derived from one classification dimension. 


Researchers address these issues when developing or choosing each specific measurement question. 
One of the purposes of pilot testing of any measurement instrument is to identify and anticipate cat- 
egorization issues. 


Appropriateness 


Appropriateness is determined at two levels: (1) the best partitioning of the data for testing hypotheses and 
showing relationships and (2) the availability of comparison data. For example, when actual age is obtained 
(ratio scale), the editor may decide to group data by age ranges to simplify pattern discovery within the 
data. The number of age groups and breadth of each range, as well as the endpoints in each range, should be 
determined by comparison data—for example, U.S. census age ranges, a customer database that includes 
age ranges, or the age data available from Fox TV used for making an advertising media buy. 


Exhaustiveness 


Researchers often add an “other” option to a measurement question because they know they cannot an- 
ticipate all possible answers. A large number of “other” responses, however, suggests the measurement 
scale the researcher designed did not anticipate the full range of information. The editor must determine 
if “other” responses appropriately fit into established categories, if new categories must be added, if 
“other” data will be ignored, or if some combination of these actions will be taken. 

Although the exhaustiveness requirement for a single variable may be obvious, a second aspect is 
less apparent. Does one set of categories—often determined before the data are collected—fully cap- 
ture all the information in the data? For example, responses to an open-ended question about family 
economic prospects for the next year may originally be categorized only in terms of being “optimistic” 
or “pessimistic.” It may also be enlightening to classify responses in terms of other concepts such as the 
precise focus of these expectations (income or jobs) and variations in responses between family heads 
and others in the family. 


Mutual Exclusivity 


Another important rule when adding categories or realigning categories is that category components 
should be mutually exclusive. This standard is met when a specific answer can be placed in one and 
only one cell in a category set. For example, in a survey, assume that you asked participants for their 
occupation. One editor’s categorization scheme might include (1) professional, (2) managerial, (3) 
sales, (4) clerical, (5) crafts, (6) operatives, and (7) unemployed. As an editor, how would you code 
a participant’s answer that specified “salesperson at Gap and full-time student” or maybe “elemen- 
tary teacher and tax preparer”? According to census data, it is not unusual for adults in our society to 
have more than one job. Here, operational definitions of the occupations categorized as “professional,” 
“managerial,” and “sales” should help clarify the situation. But the editor facing this situation also 
would need to determine how the second-occupation data are handled. One option would be to add 
a second-occupation field to the data set; another would be to develop distinct codes for each unique 
multiple-occupation combination. 
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QSR International has released version two of its qualitative research software program XSight. With tools to test theories, 
map your analysis, and report findings, XSight is used in a range of other industries that deal with unstructured information, 
including HR, law, marketing, and tourism. This screen demonstrates XSight’s “maps” function. Just like a flip chart or white 
board, “maps” give you the space to organize your thoughts and represent the connections between them visually. What 
makes XSight “maps” unique is that they can be underpinned with live links to supporting information, such as participant 
quotes. You can also import maps into your client briefs and presentations. www-.qsrinternational.com 


Single Dimension 


The problem of how to handle an occupation entry like “unemployed salesperson” brings up a fourth 
tule of category design. The need for a category set to follow a single classificatory principle means 
every option in the category set is defined in terms of one concept or construct. Returning to the 
occupation example, the person in the study might be both a salesperson and unemployed. The “sales- 
person” label expresses the concept occupation type; the response “unemployed” is another dimension 
concerned with current employment status without regard to the respondent’s normal occupation. When 
a category set encompasses more than one dimension, the editor may choose to split the dimensions 
and develop an additional data field; “occupation” now becomes two variables: “occupation type” and 
“employment status.” 


Using Content Analysis for Open Questions 


Increasingly text-based responses to open-ended measurement questions are analyzed with content 
analysis software. Content analysis measures the semantic content or the what aspect of a message. Its 
breadth makes it a flexible and wide-ranging tool that may be used as a stand-alone methodology or as 
a problem-specific technique. Trend-watching organizations like the BrainReserve, the Naisbitt Group, 
SRI International, and Inferential Focus use variations on content analysis for selected projects, often 
spotting changes from newspaper or magazine articles before they can be confirmed statistically. The 
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Naisbitt Group’s content analysis of 2 million local newspaper articles compiled over a 12-year period 
resulted in the publication of Megatrends. 


Types of Content 


Content analysis has been described as “a research technique for the objective, systematic, and quanti- 
tative description of the manifest content of a communication.” Because this definition is sometimes 
confused with simply counting obvious message aspects such as words or attributes, more recent in- 
terpretations have broadened the definition to include latent as well as manifest content, the symbolic 
meaning of messages, and qualitative analysis. One author states: 


In any single written message, one can count letters, words, or sentences. One can categorize phrases, describe 
the logical structure of expressions, ascertain associations, connotations, denotations, elocutionary forces, and one 
can also offer psychiatric, sociological, or political interpretations. All of these may be simultaneously valid. In short, a 
message may convey a multitude of contents even to a single receiver.* 


Content analysis follows a systematic process for coding and drawing inferences from texts. It starts 
by determining which units of data will be analyzed. In written or verbal texts, data units are of four 
types: syntactical, referential, propositional, or thematic. Each unit type is the basis for coding texts 
into mutually exclusive categories in our search for meaning. 


e Syntactical units can be words, phrases, sentences, or paragraphs; words are the smallest and 
most reliable data units to analyze. Although we can certainly count these units, we are more in- 
terested in the meaning their use reveals. In content analysis we might determine the words that 
are most commonly used to describe product A versus its competitor, product B. We ask, Are 
these descriptions for product A more likely to lead to favorable opinions and thus to preference 
and ultimately selection, compared to the descriptions used for product B? 


¢ Referential units are described by words, phrases, and sentences; they may be objects, events, 
persons, and so forth, to which a verbal or textual expression refers. Participants may refer to a 
product as a “classic,” a “power performer,” or “ranked first in safety”—each word or phrase 
may be used to describe different objects, and it is the object that the researcher codes and ana- 
lyzes in relation to the phrase. 


¢ Propositional units are assertions about an object, event, person, and so on. For example, a 
researcher assessing advertising for magazine subscriptions might conclude, “Subscribers who 
respond to offer A will save $15 over the single issue rate.” It is the assertion of savings that is 
attached to the text of this particular ad claim. 


¢ Thematic units are topics contained within (and across) texts; they represent higher-level abstrac- 
tions inferred from the text and its context. The responses to an open-ended question about pur- 
chase behavior may reflect a temporal theme: the past (“I never purchased an alternative brand 
before you changed the package’), the present (“I really like the new packaging’’), or the future 
(“I would buy the product more often if it came in more flavors”). We could also look at the com- 
ments as relating to the themes or topics of “packaging” versus a product characteristic, “flavors.” 


As with all other research methodologies, the analytical use of content analysis is influenced by 
decisions made prior to data collection. Content analysis guards against selective perception of the 
content, provides for the rigorous application of reliability and validity criteria, and is amenable to 
computerization. 


What Content Is Analyzed? 


Content analysis may be used to analyze written, audio, or video data from experiments, observations, 
surveys, and secondary data studies. The obvious data to be content-analyzed include transcripts of 
focus groups, transcripts of interviews, and open-ended survey responses. But researchers also use con- 
tent analysis on advertisements, promotional brochures, press releases, speeches, Web pages, historical 
documents, and conference proceedings, as well as magazine and newspaper articles. In competitive 
intelligence and the marketing of political candidates content analysis is a primary methodology. 
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Example 


Let’s look at an informal application of content analysis to a problematic open question. In this ex- 
ample, which we are processing without the use of content analysis software, suppose employees in 
the sales department of a manufacturing firm are asked, “How might company-—customer relations be 
improved?” A sample of the responses yields the following: 


¢ We should treat the customer with more respect. 


e We should stop trying to speed up the sales process when the customer has expressed objections 
or concerns. 


¢ We should have software that permits real-time tracking of a customer’s order. 


¢ Our laptops are outdated. We can’t work with the latest software or access information quickly 
when we are in the field. 


e My [the sales department] manager is rude with customers when he gets calls while I’m in the 
field. He should be transferred or fired. 


e Management should stop pressuring us to meet sales quotas when our customers have restricted 
their open-to-buy status. 


The first step in analysis requires that the units selected or developed help answer the research 
question. In our example, the research question is concerned with learning who or what the sales force 
thinks is a source for improving company-—customer relations. The first pass through the data produces 
a few general categories in one concept dimension: source of responsibility, shown in Exhibit 15-4. 
These categories are mutually exclusive. The use of “other” makes the category set exhaustive. If, how- 
ever, many of the sample participants suggested the need for action by other parties—for example, the 
government or a trade association—then including all those responses in the “other” category would 
ignore much of the richness of the data. As with coding schemes for numerical responses, category 
choices are very important. 

Since responses to this type of question often suggest specific actions, the second evaluation of 
the data uses propositional units. If we used only the set of categories in Exhibit 15-4, the analysis 
would omit a considerable amount of information. The second analysis produces categories for action 
planning: 


e Human relations. 
¢ Technology. 

e Training. 

e Strategic planning. 
e Other action areas. 


e No action area identified. 


>Exhibit 15-4 Open Question Coding Example (before revision) 
Question: “How can company-—customer relations be improved?” 
Locus of Responsibility Mentioned Not Mentioned 
A. Company 
B. Customer 
C. Joint Company-Customer 


F. Other 
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>Exhibit 15-5 Open Question Coding (after revision) 


Question: “How can company-—customer relations be improved?” 


Locus of Responsibility Frequency (n = 100) 


A. Management 


1. Sales manager 10 
2. Sales process 20 
3. Other i 
5. No action area identified 3 


C. Customer 
1. Buying processes 12 
2. Other 8 
8. No action area identified 5 
E. Technology 20 


How can we categorize a response suggesting a combined training-technology process? 
Exhibit 15-5 illustrates a combination of alternatives. Taking the categories of the first list of the 
action areas makes it possible to get an accurate frequency count of the joint classification possibilities 
for this question. 

Using available software, the researcher can spend much less time coding open-ended responses 
and capturing categories. Software also eliminates the high cost of sending responses to outside coding 
firms. What used to take a coding staff several days may now be done in a few hours. 

Content analysis software applies statistical algorithms to open-ended question responses. This per- 
mits stemming, aliasing, and exclusion processes. Stemming uses derivations of common root words to 
create aliases (e.g., using searching, searches, searched, for search). Aliasing searches for synonyms 
(wise or smart for intelligent). Exclusion filters out trivial words (be, is, the, of) in the search for 
meaning.* 

When you are using menu-driven programs, an autocategorization option creates manageable cat- 
egories by clustering terms that occur together throughout the textual data set. Then, with a few key- 
strokes, you can modify categorization parameters and refine your results. Once your categories are 
consistent with the research and investigative questions, you select what you want to export to a data 
file or in tab-delimited format. The output, in the form of tables and plots, serves as modules for your 
final report. Exhibit 15-6 shows a plot produced by a content analysis of the MindWriter complaints 
data. The distances between pairs of terms reveal how likely it is that the terms occur together, and the 
colors represent categories. 


“Don’t Know” Responses 


The “don’t know” (DK) response presents special problems for data preparation. When the DK re- 
sponse group is small, it is not troublesome. But there are times when it is of major concern, and it may 
even be the most frequent response received. Does this mean the question that elicited this response 
is useless? The answer is, It all depends. Most DK answers fall into two categories.° First, there is the 
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>Exhibit 15-6 Proximity Plot of MindWriter Customer Complaints 


Problems acquiring shipping box 


Parts not 


kept in Professionalism 
stock 


legitimate DK response when the respondent does not know the answer. This response meets our re- 
search objectives; we expect DK responses and consider them to be useful. 

In the second situation, a DK reply illustrates the researcher’s failure to get the appropriate informa- 
tion. Consider the following illustrative questions: 


Who developed the Managerial Grid concept? 
Do you believe the new president’s fiscal policy is sound? 
Do you like your present job? 


Which of the various brands of chewing gum do you believe has the best quality? 


Oo OS 


How often each year do you go to the movies? 


It is reasonable to expect that some legitimate DK responses will be made to each of these questions. 
In the first question, the respondents are asked for a level of information that they often will not have. 
There seems to be little reason to withhold a correct answer if known. Thus, most DK answers to this 
question should be considered as legitimate. A DK response to the second question presents a different 
problem. It is not immediately clear whether the respondent is ignorant of the president’s fiscal policy 
or knows the policy but has not made a judgment about it. The researchers should have asked two 
questions: In the first, they would have determined the respondent’s level of awareness of fiscal policy. 
If the interviewee passed the awareness test, then a second question would have secured judgment on 
fiscal policy. 

In the remaining three questions, DK responses are more likely to be a failure of the questioning 
process, although some will surely be legitimate. The respondent may be reluctant to give the informa- 
tion. A DK response to question 3 may be a way of saying, “I do not want to answer that question.” 
Question 4 might also elicit a DK response in which the reply translates to “This is too unimportant 
to talk about.” In question 5, the respondents are being asked to do some calculation about a topic to 
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>Exhibit 15-7 Handling “Don’t Know” Responses 


Question: Do you have a productive relationship with your present salesperson? 


Years of Purchasing Yes No Don’t Know 

Less than 1 year 10% 40% 38% 

1-3 years 30 30 382 

4 years or more 60 30 30 

Total 100% 100% 100% 
n = 650 n= 150 n = 200 


which they may attach little importance. Now the DK may mean “I do not want to do that work for 
something of so little consequence.” 


Dealing with Undesired DK Responses 


The best way to deal with undesired DK answers is to design better measurement questions. Research- 
ers should identify the questions for which a DK response is unsatisfactory and design around it. In- 
terviewers, however, often inherit this problem and must deal with it in the field. Several actions are 
then possible. First, good interviewer—respondent rapport will motivate respondents to provide more 
usable answers. When interviewers recognize an evasive DK response, they can repeat the question 
or probe for a more definite answer. The interviewer may also record verbatim any elaboration by the 
respondent and pass the problem on to the editor. 

If the editor finds many undesired responses, little can be done unless the verbatim comments can 
be interpreted. Understanding the real meaning relies on clues from the respondent’s answers to other 
questions. One way to do this is to estimate the allocation of DK answers from other data in the 
questionnaire. The pattern of responses may parallel income, education, or experience levels. Sup- 
pose a question concerning whether customers like their present salesperson elicits the answers in 
Exhibit 15-7. The correlation between years of purchasing and the “don’t know” answers and the “no” 
answers suggests that most of the “don’t knows” are disguised “no” answers. 

There are several ways to handle “don’t know” responses in the tabulations. If there are only a few, 
it does not make much difference how they are handled, but they will probably be kept as a separate 
category. If the DK response is legitimate, it should remain as a separate reply category. When we are 
not sure how to treat it, we should keep it as a separate reporting category and let the research sponsor 
make the decision. 


Missing Data 


Missing data are information from a participant or case that is not available for one or more variables 
of interest. In survey studies, missing data typically occur when participants accidentally skip, refuse 
to answer, or do not know the answer to an item on the questionnaire. In longitudinal studies, miss- 
ing data may result from participants dropping out of the study, or being absent for one or more data 
collection periods. Missing data also occur due to researcher error, corrupted data files, and changes 
in the research or instrument design after data were collected from some participants, such as when 
variables are dropped or added. The strategy for handling missing data consists of a two-step process: 
the researcher first explores the pattern of missing data to determine the mechanism for missing- 
ness (the probability that a value is missing rather than observed) and then selects a missing-data 
technique. 


389 


390 


>part IV _ Analysis and Presentation of Data 


>Exhibit 15-8 MindWriter Data Set: Missing and Out-of-Range Data 


Case 1A 1B 2A 2B 2c 
1 5.0 5.0 5.0 5.0 9.0 
2 10) 3.0 4.0 9.0 
3 5.0 5.0 5.0 5.0 5.0 
4 5.0 510 4.0 
§ 1.0 220) 
6 5.0 510 5.0 5.0 9.0 
7 5.0 5.0 5.0 5.0 5.0 
8 4.0 3.0 3.0 3.0 3.0 
g) 4.0 4.0 5.0 5:0) 5.0 
10 4.0 5.0 4.0 510 
lal 2.0 5.0 4.0 4.0 5.0 
12 6.0 4.0 3.0 3.0 4.0 
13 5.0 S10 3.0 5.0 
14 5.0 5.0 5.0 5.0 5.0 
15 5.0 4.0 5.0 5.0 4.0 
Valid 15 14 14 14 13 
Missing ) 1 4 1 2 
ean 4.53 4.50 4.45 4.14 5.61 
Range 6 2 2 3 6 
inimum 1 3 3 2 3 
aximum a ) 5 5 ©) 


Examine the sample distribution of variables from the MindWriter dataset shown in Exhibit 15-8. 
These data were collected on a 5-point interval scale. There are no missing data in variable 1A, 
although it is apparent that a range of 6 and a maximum value of 7 invalidate the calculated mean 
or average score. Variables 1B and 2B have one case missing but values that are within range. 
Variable 2A is missing four cases, or 27 percent of its data points. The last variable, 2C, has a 
range of 6, two missing values, and three values coded as “9.” A “9” is often used as a DK or 
missing-value code when the scale has a range of less than 9 points. In this case both blanks and 
9s are present—a coding concern. Notice that the fifth respondent answered only two of the five 
questions and the second respondent had two miscoded answers and one missing value. Finally, 
using descriptive indexes of shape, discussed in Appendix 15a, you can find three variables that 
depart from the symmetry of the normal distribution. They are skewed (or pulled) to the left by a 
disproportionately small number of 1s and 2s. And one variable’s distribution is peaked beyond 
normal dimensions. We have just used the minimum and maximum values, the range, and the 
mean and have already discovered errors in coding, problems with respondent answer patterns, 
and missing cases. 
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Mechanisms for Dealing with Missing Data 


By knowing what caused the data to be missing, the researcher can select the appropriate missing- 
data technique and thus avoid introducing bias in subsequent analysis. There are three basic types of 
missing data: 


e Data missing completely at random (MCAR)—the probability that a particular variable is miss- 
ing is NOT dependent on the variable itself and is NOT dependent on another variable in the 
data set (e.g., a participant inadvertently skips a question). 


Data missing at random (MAR)—the probability that a particular variable is missing is NOT 
dependent on the variable itself but is dependent on another variable in the data set (e.g., the 
answer to the first question of a branched-question set might cause missing data to the second 
question within the branched-question set). 


Data missing but not missing at random (NMAR)—when missing data are not predictable from 
other variables in the data set. 


Three techniques are used to salvage data sets with missing data: 


° Listwise deletion—cases with missing data on one variable are deleted from the sample for all 
analyses of that variable. 


¢ Pairwise deletion—missing data are estimated using all cases that have data for each variable or 
pair of variables; the estimation replaces the missing data. 


¢ Predictive replacement—missing data are predicted from observed values on another variable; 
the observed value is used to replace the missing data. 


Listwise deletion is the default option for MCAR and is used by most statistical packages like SPSS 
and SAS. No bias is introduced because only fully complete cases are used as a sample for the variable. 
However, if data are MAR, not MCAR, then bias may be introduced, especially if a large number of 
cases are omitted from the sample. For example, in a survey if men were more likely than women to be 
responsible for missing data on the variable relating to training preference, then deleting men from the 
sample biases any analysis of training preferences toward women’s training preference. 

Pairwise deletion assumes data are MCAR. This technique, although used historically by linear 
models, has the potential for introducing bias. 

Predictive replacement assumes data are MAR. One common option available on most statistical 
packages is the use of a mean or other central tendency score as the replacement for the missing data. 
This practice reduces the variability of the data, which can introduce bias. 

When data are NMAR, the missing data are factored into the analysis as a separate category of data 
on that variable. 


> Data Entry 


Data entry converts information gathered by secondary or primary methods to a medium for viewing 
and manipulation. Keyboarding remains a mainstay for researchers who need to create a data file im- 
mediately and store it in a minimal space on a variety of media. However, researchers have profited 
from more efficient ways of speeding up the research process, especially from bar coding and optical 
character and mark recognition. 


Alternative Data Entry Formats 
Keyboarding 


A full-screen editor, with which an entire data file can be edited or browsed, is a viable means of data 
entry for statistical packages like SPSS or SAS. SPSS offers several data entry products, including 
Data Entry Builder™, which enables the development of forms and surveys, and Data Entry Station™, 
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>Exhibit 15-9 Data Fields, Records, Files, and Databases 


Data fields represent single elements of information (e.g., an answer to a particular question) from all participants in a study. Data fields 
can contain numeric, alphabetic, or symbolic information. A data record is a set of data fields that are related to one case or participant 
(e.g., the responses to one completed survey). Records represent rows in a data file or spreadsheet program worksheet. Data files are 
sets of records (e.g., responses from all participants in a single study) that are grouped together for storage on diskettes, disks, tapes, 
CD-ROM, or optical disks. Databases are made up of one or more data files that are interrelated. A database might contain all customer 
survey information collected quarterly for the past 10 years. 


Data Field 
Study 1 
© Study 2 
L Study 3 
Database Stored on 
Zenith : Data Record Electronic Media 


Surveys or Data File 
Instruments 


which gives centralized entry staff, such as telephone interviewers or online participants, access to the 
survey. Both SAS and SPSS offer software that effortlessly accesses data from databases, spreadsheets, 
data warehouses, or data marts. 


Database Development For large projects, database programs serve as valuable data entry de- 
vices. A database is a collection of data organized for computerized retrieval. Programs allow users to 
define data fields and link files so that storage, retrieval, and updating are simplified. The relationship 
between data fields, data records, files, and databases is illustrated in Exhibit 15-9. A company’s or- 
ders serve as an example of a database. Ordering information may be kept in several files: salesperson’ s 
customer files, customer financial records, order production records, and order shipping documenta- 
tion. The data are separated so that authorized people can see only those parts pertinent to their needs. 
However, the files may be linked so that when, say, a customer changes his or her shipping address, 
the change is entered once and all relevant files are updated. Another database entry option is e-mail 
data capture. It has become popular with those using e-mail-delivered surveys. The e-mail survey can 
be delivered to a specific respondent whose e-mail address is known. Questions are completed on the 
screen, returned via e-mail, and incorporated into a database.° An intranet can also capture data. When 
participants linked by a network take an online survey by completing a database form, the data are cap- 
tured in a database in a network server for later or real-time analysis.’ ID and password requirements 
can keep unwanted participants from skewing the results of an online survey. 

Researchers consider database entry when they have large amounts of potentially linked data that will 
be retrieved and tabulated in different ways over time. Another application of a database program is as a 
“front-end” entry mechanism. A telephone interviewer may ask the question “How many children live in 
your household?” The computer’s software has been programmed to accept any answer between 0 and 
20. If a “P” is accidentally struck, the program will not accept the answer and will return the interviewer 
to the question. With a precoded online instrument, some of the editing previously discussed is done by 
the program. In addition, the program can be set for automatic conditional branching. In the example, an 
answer of | or greater causes the program to prompt the questioner to ask the ages of the children. A 0 
causes the age question to be automatically skipped. Although this option is available whenever interac- 
tive computing is used, front-end processing is typically done within the database design. The database 
will then store the data in a set of linked files that allow the data to be easily sorted. Descriptive statistics 
and tables—the first steps in exploring data—are readily generated from within the database. 


Spreadsheet Spreadsheets are a specialized type of database for data that need organizing, tabu- 
lating, and simple statistics. They also offer some database management, graphics, and presentation 
capabilities. Data entry on a spreadsheet uses numbered rows and lettered columns with a matrix 
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>Exhibit 15-10 Data Entry Using Spreadsheets 


Each row is a record (a single participant’s responses). Each column is a variable measured in the survey. In this 
survey, questions 1, 3, and 5 are nominal variables that have two response categories. Question 6 uses multiple 
columns because it is a multipart rating question using a 1-to-5 scale. This is a typical way of coding variables in a 
spreadsheet before they are imported by SPSS (assuming you are using a spreadsheet instead of the SPSS Data 
Editor to start your study). Note that each participant is assigned an identification number (case ID). After running pre- 
liminary frequencies, having a case ID data field enables you to quickly find and correct suspect data like odd value 
codes or missing cases. 


& Microsoft Excel - Sample Survey Spreadsheet 
File Edit View Insert Format Tools Data Window Help 


Deeeaa 6RY eB Sone Bx~ he 43) Mo[Bjio~ -B. 
Arial 74 + BLUE BES %, BB Ee B- o-A-, 
K21 ¥ fe 
| A B Cc D E F G H I A K L 

1 Case ID| Q1 | @2 | a3 | a4 | 25 | Q6a | Q6b | Q6c | O6d | AGe | AEF 
2) 0001 1 2 1] 10 2 1 2 1 1 4 
3/0002 2 5 2 7 1 2 2 3 2 5 
4. 0003 1 2 1 6 2 2 4 3 4 4 
5| 0004 1 2 1 1 1 3 4 4 4 4 
6| 0005 2 6 2 8 2 3 5 4 2| 5 1 
7 0006 2 1 2 8 2 3 A a) ee) 1 
a| 0007 1 3 4 8 1 2 5 3 5 3 
30008 2 4 2 5 2 3 3 4 5 3 
10, 0009 1 2 1 9 1 3 2 4 5 5 
11,0010 2 2 2 9 2 4 2 5 5 5 
12, 0014 2 5 2 9 1 4 1 1 3] | 5 
13,0012 1 2 1 9 1 2 2, 2] Sa} Sd 2 
14, 0013 2 1 2 3 2 5 3 3 4 1 
15.0014 1 6 1 2 2 3 4 4 5 2 
160015 2 4 2 3 1 1 4 3 1 3 
17,| 0016 2 3 2 4 2 5 5 5 2| 5 4 
18, 0017 1 3 1 6 1 5 5 2 1 1 4 
19, 0018 2 3 2 5 2 5 5 2 2 2 3 


of thousands of cells into which an entry may be placed. Spreadsheets allow you to type numbers, 
formulas, and text into appropriate cells. Many statistics programs for personal computers and also 
charting and graphics applications have data editors similar to the Excel spreadsheet format shown in 
Exhibit 15-10. This is a convenient and flexible means for entering and viewing data. 


Optical Recognition 


If you use a PC image scanner, you probably are familiar with optical character recognition (OCR) 
programs that transfer printed text into computer files in order to edit and use it without retyping. There 
are other related applications. Optical scanning of instruments—the choice of testing services—is 
efficient for researchers. Examinees darken small circles, ellipses, or spaces between sets of paral- 
lel lines to indicate their answers. A more flexible format, optical mark recognition (OMR) uses 
a spreadsheet-style interface to read and process user-created forms. Optical scanners process the 
marked-sensed questionnaires and store the answers in a file. This method, most often associated with 
standardized and preprinted forms, has been adopted by researchers for data entry and preprocessing 
due to its speed (10 times faster than keyboarding), cost savings on data entry, convenience in charting 
and reporting data, and improved accuracy. It reduces the number of times data are handled, thereby 
reducing the number of errors that are introduced. 

Other techniques include direct-response entry, of which voting procedures used in several states 
are an example. With a specially prepared punch card, citizens cast their votes by pressing a pen-shaped 
instrument against the card next to the preferred candidate. This opens a small hole in a specific col- 
umn and row of the card. The cards are collected and placed directly into a card reader. This method 
also removes the coding and entry steps. Another governmental application is the 1040EZ form used 
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Seeking Clean Netnography Data 


In content analysis, researchers analyze anecdotal informa- 
tion to determine the major themes of comments. Research- 
ers are increasingly using feedback data posted on the Web or 
internally in intranet chat rooms to capture such content. Postings 
come in the form of product reviews, company evaluations, em- 
ployee experiences, message board and newsgroup postings, 
as well as chat room dialogues and discussion forum postings. 
Selecting such data for analysis takes some careful screen- 
ing. When using net postings, Chrysanthos Dellarocas of 
Massachusetts Institute of Technology recommends that the 
researcher extract information from sites that are able to authen- 
ticate the identity of participants; this reduces or prevents unfair 
ratings and discriminatory behavior. Some opinion sites use fi- 
nancial incentives to encourage postings, which may encourage 


the posting of numerous comments from a single individual, thus 
introducing bias. Dina Mayzlin of the Yale School of Manage- 
ment suggests selecting comments from a larger number of 
sites, rather than selecting large numbers of comments from a 
single site; such dispersion reduces bias. In cleaning such data, 
Zhilin Yang of City University of Hong Kong and Robin Peterson 
at New Mexico State University suggest that researchers screen 
messages to detect irrelevant anecdotes, duplication, requests 
for messages to be posted, and spam messages, all of which if 
left in the data set would inject bias. 

If you would like to try collecting and screening such data, 
some useful sites include ZDnet.com, Bizrate.com, Amazon.com, 
eBay.com, Elance.com, Complaints.com, reviewcentre.com, 
and Epinion.com. 


by the Internal Revenue Service. It is designed for computerized number and character recognition. 
Similar character recognition techniques are employed for many forms of data collection. Again, both 
approaches move the response from the question to data analysis with little handling. 


Voice Recognition 


The increase in computerized random dialing has encouraged other data collection innovations. Voice 
recognition and voice response systems are providing some interesting alternatives for the telephone 
interviewer. Upon getting a voice response to a randomly dialed number, the computer branches into 
a questionnaire routine. These systems are advancing quickly and will soon translate recorded voice 
responses into data files. 


Digital 


Telephone keypad response, frequently used by restaurants and entertainment venues to evaluate cus- 
tomer service, is another capability made possible by computers linked to telephone lines. Using the 
telephone keypad (touch-tone), an invited participant answers questions by pressing the appropriate 
number. The computer captures the data by decoding the tone’s electrical signal and storing the nu- 
meric or alphabetic answer in a data file. Although not originally designed for collecting survey data, 
software components within Microsoft Windows 7 have advanced speech recognition functionality, 
enabling people to enter and edit data by speaking into a microphone.® 

Field interviewers can use mobile computers or notebooks instead of clipboards and pencils. With 
a built-in communications modem, wireless LAN (or local area network), or cellular link, their files 
can be sent directly to another computer in the field or to a remote site (cloud). This lets supervisors in- 
spect data immediately or simplifies processing at a central facility. This is the technology that Nielsen 
Media is using with its portable PeopleMeter. 


Bar Code Since adoption of the Universal Product Code (UPC) in 1973, the bar code has devel- 
oped from a technological curiosity to a business mainstay. After a study by McKinsey & Company, 
the Kroger grocery chain pilot-tested a production system and bar codes became ubiquitous in that 
industry.’ 
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Bar-code technology is used to simplify the interviewer’s role as a data recorder. When an inter- 
viewer passes a bar-code wand over the appropriate codes, the data are recorded in a small, lightweight 
unit for translation later. In the large-scale processing project Census 2000, the Census Data Capture 
Center used bar codes to identify residents. Researchers studying magazine readership can scan bar 
codes to denote a magazine cover that is recognized by an interview participant. 

The bar code is used in numerous applications: point-of-sale terminals, hospital patient ID brace- 
lets, inventory control, product and brand tracking, promotional technique evaluation, shipment track- 
ing, marathon runners, rental car locations (to speed the return of cars and generate invoices), and 
tracking of insects’ mating habits. The military uses two-foot-long bar codes to label boats in storage. 
The codes appear on business documents, truck parts, and timber in lumberyards. Federal Express 
shipping labels use a code called Codabar. Other codes, containing letters as well as numbers, have 
potential for researchers. 


On the Horizon 


Even with these time reductions between data collection and analysis, innovations offer significant 
promise. The capability to integrate visual images, streaming video, audio, and data has replaced video 
equipment as the preferred method for recording an experiment, interview, or focus group. Response 
data can be extracted for data analysis, while the audio and visual images remain intact for later evalu- 
ation. Although technology will never replace researcher judgment, it can reduce data-handling errors, 
decrease time between data collection and analysis, and help provide more usable information. 
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1 The first step in data preparation is to edit the collected raw 


data to detect errors and omissions that would compromise 
quality standards. The editor is responsible for making sure 

the data are accurate, consistent with other data, uniformly 

entered, and ready for coding. In survey work, it is common 
to use both field and central editing. 


2 Coding is the process of assigning numbers and other 


symbols to answers so that we can classify the responses 
into categories. Categories should be appropriate to the re- 
search problem, exhaustive of the data, mutually exclusive, 
and unidimensional. The reduction of information through 
coding requires that the researcher design category sets 
carefully, using as much of the data as possible. Codebooks 
are guides to reduce data entry error and serve as a com- 
pendium of variable locations and other information for the 
analysis stage. Software developments in survey construc- 
tion and design include embedding coding rules that screen 
data as they are entered, identifying data that are not en- 
tered correctly. 


Closed questions include scaled items and other items 
for which answers are anticipated. Precoding of closed 
items avoids tedious completion of coding sheets for 
each response. Open-ended questions are more difficult 
to code since answers are not prepared in advance, but 
they do encourage disclosure of complete information. A 
systematic method for analyzing open-ended questions 


is content analysis. It uses preselected sampling units to 
produce frequency counts and other insights into data 
patterns. 


“Don’t know” replies are evaluated in light of the question’s 
nature and the respondent. While many DKs are legitimate, 
some result from questions that are ambiguous or from 

an interviewing situation that is not motivating. It is bet- 

ter to report DKs as a separate category unless there are 
compelling reasons to treat them otherwise. Missing data 
occur when respondents skip, refuse to answer, or do not 
know the answer to a questionnaire item, drop out of the 
study, or are absent for one or more data collection periods. 
Researcher error, corrupted data files, and changes to the 
instrument during administration also produce missing data. 
Researchers handle missing data by first exploring the data 
to discover the nature of the pattern and then selecting a 
suitable technique for replacing values by deleting cases 

(or variables) or estimating values. 


Data entry is accomplished by keyboard entry from pre- 
coded instruments, optical scanning, real-time keyboarding, 
telephone pad data entry, bar codes, voice recognition, 
OCR, OMR, and data transfers from electronic notebooks 
and laptop computers. Database programs, spreadsheets, 
and editors in statistical software programs offer flexibility 
for entering, manipulating, and transferring data for analysis, 
warehousing, and mining. 


396 >part IV Analysis and Presentation of Data 


er 


bar code 395 data preparation 376 optical mark recognition (OMR) 393 
codebook 380 data record 392 optical scanning 393 
coding 379 database 392 precoding 381 
content analysis 384 “don’t know” (DK) response 387 spreadsheet 392 
data entry 391 editing 377 voice recognition 394 
data field 392 missing data 389 
data file 392 optical character recognition (OCR) 393 
>dal ion lon 
Terms in Review d Keep a close watch on your stock, and mark down when 
1. Define or explain: you have to—that is, rather than wait, take a small mark- 
a Coding rules. down on a shoe that is not moving at the time. 
b Spreadsheet data entry. © Using the SM ened, 
c Barcodes. f Less advance buying—more dependence on in-stock 
shoes. 
d Precoded instruments. 
: g Sales—catch bad guys before it’s too late and close 
e Content analysis. - 
f Missing data. 


h Buy as much good merchandise as you can at special 
Optical mark recognition. prices to help make up some markdowns. 
2 How should the researcher handle “don’t know” responses? 


Ko} 


i Reducing opening buys and depending on fill-in service. 


PMs for salespeople. 
Making Research Decisions vee 


3 Aproblem facing shoe store managers is that many shoes j Buy more frequently, better buying, PMs on slow-moving 
eventually must be sold at markdown prices. This prompts merchandise. 
us to conduct a mail survey of shoe store managers in k Careful buying at lowest prices. Cash on the buying line. 
which we ask, What methods have you found most suc- Buying closeouts, FDs, overstock, “cancellations.” (FD 
cessful for reducing the problem of high markdowns? We stands for “factory-discontinued” style.) 


are interested in extracting as much information as possible | 
from these answers to better understand the full range of 
strategies that store managers use. Establish what you think 
are category sets to code 500 responses similar to the 14 
given here. Try to develop an integrated set of categories 


that reflects your theory of markdown management. After 
developing the set, use it to code the 14 responses. n_ No successful method with the current style situation. 


Manufacturers are experimenting, the retailer takes the 
markdowns—cuts gross profit by about 3 percent—keep 
your stock at lowest level without losing sales. 


By buying less “chanceable” shoes. Buy only what you 
need, watch sizes, don’t go overboard on new fads. 


m Buying more staple merchandise. Buying more from 
fewer lines. Sticking with better nationally advertised 
merchandise. 


a Have not found the answer. As long as we buy style 
shoes, we will have markdowns. We use PMs on slow 
merchandise, but it does not eliminate markdowns. (PM 
stands for “push-money” —special item bonuses for sell- 4 Select a small sample of class members, work associ- 
ing a particular style of shoe.) ates, or friends and ask them to answer the following in 

a paragraph or two: What are your career aspirations 

for the next five years? Use one of the four basic units 

of content analysis to analyze their responses. Describe 


your findings as frequencies for the unit of analysis 
c By putting PMs on any slow-selling items and promoting selected. 


same. More careful check of shoes purchased. 


b Using PMs before too old. Also reducing price during 
season. Holding meetings with salespeople indicating 
which shoes to push. 


Bringing Research to Life 
5 What data preparation process was Jason doing during 
data entry? 


6 Data entry followed data collection in the research profiled 
during the opening vignette. What concerned Jason about 
this process? 


From Concept to Practice 

7 Choose one of the cases from the text website that has an 
instrument (check the Case Abstracts section for a listing of 
all cases and an abstract for each). Code the instrument for 
data entry. 
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From the Headlines 

8 Simplement, Inc., which provides data sourcing solutions for 
Systems Applications and Products (SAP) customers, an- 
nounced the achievement of subsecond response times for 
discovery and exploration in billion row datasets involving sales 
data collected in the sales and distribution module of SAP. 
Business users employing technologies like Microsoft Excel 
and Tableau can immediately explore and visualize supersized 
data sets leading to higher productivity and more meaningful 
insights. What other applications would you recommend to 
your organization, particularly marketing or IT staffs? 


eT 


Inquiring Minds Want to Know—NOW! 


Mastering Teacher Leadership 


NCRCC: Teeing Up and New Strategic Direction 


NetConversions Influences Kelley Blue Book 


* You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine whether 
a case provides data, the research instrument, video, or other supplementary material. Written cases are downloadable from the 
text website (www.mhhe.com/cooper1 2e). All video material and video cases are available from the Online Learning Center. 


Describing Data Statistically 


In the first part of Chapter 15, we discussed how responses 
from participants are edited, coded, and entered. Creating 
numerical summaries of this process provides valuable in- 
sights to analysts about their effectiveness. In this appen- 
dix, we review concepts from your introductory statistics 
course that offer descriptive tools for cleaning data, dis- 
covering problems, and summarizing distributions. A dis- 
tribution (of data) is an array of value counts from lowest 
to highest value of a variable, resulting from the tabulation 
of incidence. Descriptive statistical measures are used to 
depict the center, spread, and shape of distributions and 
are helpful as preliminary tools for data description. We 
will define these measures and describe their use as de- 
scriptive statistics after introducing a sample data set and 
an overview of basic concepts. 


Reviewing Statistical Concepts 


The LCD (liquid crystal display) TV market is an interest- 
ing market to watch because of the changes in technology 
and marketing. Currently the major players in this market 
are Sharp, LG Electronics/Zenith, Samsung, Sony, Dell, 
and Panasonic. Only a few other brands earn a noticeable 
market share. Sharp products currently represent the larg- 
est percentage of unit sales. Let’s assume we are inter- 
ested in evaluating annual unit sales increases of several 
manufacturers. We survey nine manufacturers and we find 
a frequency distribution (an ordered array of all values for 
a variable) of annual percentage of unit sales increases: 5, 
6, 6, 7, 7, 7, 8, 8, 9. From these unit sales scores, we con- 
struct a table for arraying the data. It presents value codes 
from lowest to highest value, with columns for count, per- 
cent, percent for missing values, and cumulative percent. 
An example is presented in Exhibit 15a-1. 

The table arrays data by assigned numerical value, in 
this case the actual percentage unit sales increase recorded 
(far-left column). To discover how many manufacturers 
were in each unit sales increase category, you would read 
the frequency column. For example, at the intersection of 
the frequency column and the second row, there are two 
companies that posted a 6 percent annual unit sales in- 
crease. In the percentage column, you see what percentage 
of TV manufacturers in the survey gave a response for each 
level of unit sales increase. The three manufacturers who 
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had unit sales increases of 7 percent represent 33.3 per- 
cent of the total number of manufacturers surveyed 
(3/9 * 100). The cumulative percentage reveals the num- 
ber of manufacturers that provided a response and any 
others that preceded it in the table. For this example, LCD 
TV percentage unit sales increases between 5 and 7 per- 
cent represent 66.7 percent. The cumulative percentage 
column is helpful primarily when the data have an under- 
lying order. If, in part B, we create a code for source of 
origin (foreign = 1, domestic = 2) to each of the nine 
LCD TV manufacturers, the cumulative percentage col- 
umn would provide the proportion. The proportion is the 
percentage of elements in the distribution that met a crite- 
rion. In this case, the criterion is the origin of manufacture. 

In Exhibit 15a-2, the bell-shaped curve that is super- 
imposed on the distribution of annual unit sales increases 
(percent) for LCD TV manufacturers is called the normal 
distribution. The distribution of values for any variable that 
has a normal distribution is governed by a mathematical 
equation. This distribution is a symmetrical curve and re- 
flects a frequency distribution of many natural phenomena 
such as the height of people of a certain gender and age. 

Many variables of interest that researchers will mea- 
sure will have distributions that approximate a standard 
normal distribution. A standard normal distribution is a 
special case of the normal distribution in which all values 
are given standard scores. This distribution has a mean of 
0 and a standard deviation of 1. For example, a manufac- 
turer that had an annual unit sales increase of 7 percent 
would be given a standard score of zero since 7 is the mean 
of the LCD TV distribution. A standard score (or Z score) 
tells you how many units a case (a manufacturer in this 
example) is above or below the mean. The Z score, being 
standardized, allows us to compare the results of differ- 
ent normal distributions, something we do frequently in 
research. Assume that Zenith has an annual unit sales in- 
crease of 9 percent. To calculate a standard score for this 
manufacturer, you would find the difference between the 
value and the mean and divide by the standard deviation 
of the distribution shown in Exhibit 15a-1. 


“a1 —_ Value —- Mean _9-7 
Zenith s Sandee ACOnE = iad deviation D2 
= 1.64 
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>Exhibit 15a-1 Annual Percentage Unit Sales Increases for LCD TV Manufacturers 


A 
Unit Sales Increase (%) Frequency Percentage Cumulative Percentage 
5 1 Wel allel 
6 2 22e2 33.3 
uf 3 33.3 66.7 
8 2 22.2 88.9 
9 1 Well 100.0 
Total 9) 100.0 
B 
Company Origin Unit Sales Increase (%) Frequency Percentage Cumulative Percent 
Origin, foreign (1) 6 1 Hla Waal 
w 2 222 33.3 


>Exhibit 15a-2 Histogram of Annual Unit Sales Increase (%) 
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>Exhibit 15a-3 Characteristics of Distributions 


Shape: 


Skewness Normal Symmetric 


Positive 
or Right Skewed 


Negative 
or Left Skewed 


95% 
68% 


—20 -lo0 pw +10 +20 


Mean Median 
Mode 


Location 


Mode | Mean 
Median 


Mean 
Median 


A 


B 


Cc 


The standard normal distribution, shown in part A of 
Exhibit 15a-3, is a standard of comparison for describing 
distributions of sample data. It is used with inferential sta- 
tistics that assume normally distributed variables. 

We will come back to this exhibit in a moment. Now 
let’s review some descriptive tools that reveal the impor- 
tant characteristics of distributions. The characteristics of 
central tendency, variability, and shape are useful tools 
for summarizing distributions. Their definitions, applica- 
tions, and formulas fall under the heading of descriptive 
statistics. The definitions will be familiar to most readers. 


Measures of Central Tendency 


Summarizing information such as that from our collected 
data of LCD TV manufacturers often requires the descrip- 
tion of “typical” values. Suppose we want to know the 
typical percentage unit sales increase for these companies. 
We might define typical as the average response (mean); 
the middle value, when the distribution is sorted from low- 
est to highest (median); or the most frequently occurring 


Ne! 
Shape: : ' F 
Kurtosis Mesokurtic Leptokurtic Platykurtic 
D E F 


value (mode). The common measures of central tendency 
(or center) include the mean, median, and mode. 
The mean is calculated by the following formula: 


__ ok 


x= 


For the unit sales increase variable, the distribution of re- 
sponses is 5, 6, 6, 7, 7, 7, 8, 8, 9. The arithmetic average, 
or mean (the sum of the nine values divided by 9), is 


SOP GF TE eS + Bes tO 
g 


= 7 (an average 7% unit sales increase) 


The median is the midpoint of the distribution. Half of 
the observations in the distribution fall above and the other 
half fall below the median. When the distribution has an 
even number of observations, the median is the average of 
the two middle scores. The median is the most appropri- 
ate locator of center for ordinal data and has resistance to 
extreme scores, thereby making it a preferred measure for 


interval and ratio data when their distributions are not nor- 

mal. The median is sometimes symbolized by M or mdn. 
From the sample distribution for the percentage unit 

sales increase variable, the median of the nine values is 7: 


566 777 88 9 


If the distribution had 10 values, the median would be the 
average of the values for the fifth and sixth cases. 

The mode is the most frequently occurring value. There 
may be more than one mode in a distribution. When there 
is more than one score that has the highest yet equal fre- 
quency, the distribution is bimodal or multimodal. There 
may be no mode in a distribution if every score has an 
equal number of observations. The mode is the location 
measure of central tendency for nominal data and a point 
of reference along with the median and mean for examin- 
ing spread and shape of distributions. In our LCD TV per- 
centage unit sales increase example, the most frequently 
occurring value is 7. As revealed in the frequency distribu- 
tion in Exhibit 15a-2, there are three companies that have 
unit sales increases of 7 percent. 

Notice in Exhibit 15a-3, part A, that the mean, median, 
and mode are the same in a normal distribution. When 
these measures of central tendency diverge, the distribu- 
tion is no longer normal. 


Measures of Variability 


The common measures of variability, alternatively re- 
ferred to as dispersion or spread, are the variance, stan- 
dard deviation, range, interquartile range, and quartile 
deviation. They describe how scores cluster or scatter in 
a distribution. 

The variance is a measure of score dispersion about the 
mean. If all the scores are identical, the variance is 0. The 
greater the dispersion of scores, the greater the variance. 
Both the variance and the standard deviation are used with 
interval and ratio data. The symbol for the sample variance 
is s*, and that for the population variance is the Greek let- 
ter sigma, squared (o)’. The variance is computed by sum- 
ming the squared distance from the mean for all cases and 
dividing the sum by the total number of cases minus |: 


Sum of the squared distances 
from mean for all cases 


. —_— a 
Variance = s (Number of cases — 1) 


Dix,- xy 


n= | 


2 


For the percentage unit sales increase variable, we would 
compute the variance as 


(5-72 + (6-72 + (6-79 
+(7-T8+(-72+ (70-79 
,. +8-72+8-72+ 0-7" 


a [eel 
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The standard deviation summarizes how far away from 
the average the data values typically are. It is perhaps the 
most frequently used measure of spread because it im- 
proves interpretability by removing the variance’s square 
and expressing the deviations in their original units (e.g., 
sales in dollars, not dollars squared). It is also an impor- 
tant concept for descriptive statistics because it reveals the 
amount of variability within the data set. Like the mean, 
the standard deviation is affected by extreme scores. The 
symbol for the sample standard deviation is s, and that for 
a population standard deviation is o. Alternatively, it is 
labeled std. dev. You can calculate the standard deviation 
by taking the square root of the variance: 


s=Vi 


The standard deviation for the percentage unit sales in- 
crease variable in our example is 1.22: 


122=V15 


The range is the difference between the largest and 
smallest scores in the distribution. The percentage annual 
unit sales increase variable has a range of 4 (9 — 5 = 4). 
Unlike the standard deviation, the range is computed from 
only the minimum and maximum scores; thus, it is a very 
rough measure of spread. With the range as a point of 
comparison, it is possible to get an idea of the homogene- 
ity (small std. dev.) or heterogeneity (large std. dev.) of 
the distribution. For a homogeneous distribution, the ratio 
of the range to the standard deviation should be between 
2 and 6. A number above 6 would indicate a high degree 
of heterogeneity. In the percentage unit sales increase ex- 
ample, the ratio is 4/1.22 = 3.28. The range provides use- 
ful but limited information for all data. It is mandatory for 
ordinal data. 

The interquartile range (IQR) is the difference between 
the first and third quartiles of the distribution. It is also 
called the midspread. Ordinal or ranked data use this mea- 
sure in conjunction with the median. It is also used with 
interval and ratio data when asymmetrical distributions are 
suspected or for exploratory analysis. Recall the follow- 
ing relationships: the minimum value of the distribution is 
the O percentile; the maximum, the 100th percentile. The 
first quartile (Q,) is the 25th percentile; the median, Q,, 
is the 50th percentile. The third quartile (Q,) is the 75th 
percentile. For the percentage unit sales increase data, the 
quartiles are 


5.6 6.7.7 7.8% § 
Qa 2 @ dQ, 


The quartile deviation, or semi-interquartile range, is ex- 
pressed as 
Q, = QO, 
Q= 2 


The quartile deviation is always used with the median for 
ordinal data. It is helpful for interval and ratio data when 
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the distribution is stretched (or skewed) by extreme val- 
ues. In a normal distribution, the median plus one quartile 
deviation (Q) on either side encompasses 50 percent of 
the observations. Eight Qs cover approximately the range. 
Q’s relationship with the standard deviation is constant 
(Q = .6745s) when scores are normally distributed. For 
our annual percentage unit sales increase example, the 
quartile deviation is 1 [(6 — 8)/2 = 1]. 


Measures of Shape 


The measures of shape, skewness and kurtosis, describe 
departures from the symmetry of a distribution and its 
relative flatness (or peakedness), respectively. They use 
deviation scores (X — X). Deviation scores show us how 
far any observation is from the mean. The company that 
posted a percentage unit sales increase of 9 has a devia- 
tion score of 2 (9 — 7). The measures of shape are often 
difficult to interpret when extreme scores are in the dis- 
tribution. Generally, shape is best communicated through 
visual displays. (Refer to the graphics in Exhibit 15a-3, 
parts B through F.) From a practical standpoint, the calcu- 
lation of skewness and kurtosis is easiest with spreadsheet 
or statistics software. 

Skewness is a measure of a distribution’s deviation from 
symmetry. In a symmetrical distribution, the mean, me- 
dian, and mode are in the same location. A distribution that 
has cases stretching toward one tail or the other is called 
skewed. As shown in Exhibit 15a-3, part B, when the 
tail stretches to the right, to larger values, it is positively 
skewed. In part C, scores stretching toward the left, toward 
smaller values, skew the distribution negatively. Note 
the relationship between the mean, median, and mode in 
asymmetrical distributions. The symbol for skewness is sk. 

n x,-xP 
sk = (n — 1)(n — 2) yl 
where s is the sample standard deviation (the unbiased 
estimate of sigma). 


When a distribution approaches symmetry, sk is ap- 
proximately 0. With a positive skew, sk will be a positive 
number; with negative skew, sk will be a negative num- 
ber. The calculation of skewness for our annual percent- 
age unit sales increase data produces an index of 0 and 
reveals no skew. 

As illustrated in the lower portion of Exhibit 15a-3, 
kurtosis is a measure of a distribution’s peakedness (or 
flatness). Distributions that have scores that cluster heav- 
ily or pile up in the center (along with more observations 
than normal in the extreme tails) are peaked or leptokurtic. 
Flat distributions, with scores more evenly distributed and 
tails fatter than a normal distribution, are called playkur- 
tic. Intermediate or mesokurtic distributions approach 
normal—neither too peaked nor too flat. The symbol for 
kurtosis is ku. 


ku = 


n(n+1) (5) 
(n — 1)(n — 2)(n — 3) s 

3(n — 1) 
~ (n—2)(n—3) 


where s is the sample standard deviation (the unbiased es- 
timate of sigma). 

The value of ku for a normal or mesokurtic distribution 
is close to 0. A leptokurtic distribution will have a posi- 
tive value, and the playkurtic distribution will be nega- 
tive. As with skewness, the larger the absolute value of 
the index, the more extreme is the characteristic. In the 
annual percentage unit sales increase example, the kurto- 
sis is calculated as —0.29, which suggested a very slight 
deviation from a normally shaped curve with some flat- 
tening contributed by smaller-than-expected frequencies 
of the value 7 in the example distribution. 
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learningobjectives 


After reading this chapter, you should understand .. . 


That exploratory data analysis techniques provide insights and data diagnostics by emphasizing visual 
representations of the data. 


How cross-tabulation is used to examine relationships involving categorical variables, serves as a 
framework for later statistical testing, and makes table-based analysis using one or more control 
variables an efficient tool for data visualization and decision making. 


6 6On a day-to-day basis, look for inspiration and 
ideas outside the research industry to influence 
your thinking. For example, data visualization 
could be inspired by an infographic you see ina 


favorite magazine, or even a piece of art you see 
in a museum. 9 9 


Amanda Durkee, partner 
Zanthus 


bringingresearchtolife ...<° 


“Sammye Grayson, meet Myra Wines from Mind- 
Writer. We'll be working with her on a short-turnaround 
project during the next week.” Sammye rises to shake 
Myra’s extended hand, as Jason asks, innocently, 
“Anything interesting on those initial cross-tabs?” 

Myra smiles, raises an expressive eyebrow, and 
waits for Sammye’s response. 

Sammye hesitates and then, looking at Jason 
for some signal of why he asked the question in the 
presence of a different client, responds, “Three of the 
early cross-tabs appeared to show some support for 
the board’s assumptions about the alcohol issue—on 
whether current patrons endorse the selling of beer and 
wine during intermissions. But we’re not far enough 
into the data to say which of the board’s assumptions 
are fully correct and which might have to be modified 
based on the patterns emerging within subgroups of the 
sample.” 

Jason raises a hand to stop the detailed answer to his 
question. Sammye knows from the look on his face that 
she’s done something wrong. 

“T shouldn’t have answered your question,” blurts 
Sammye. “I walked right into the trap you set, eyes 
wide open.” 

Myra jumps in before Jason can respond. “I’ve seen 
Jason do this once before to an intern, so you should 
feel like one of the team. And, no, you shouldn’t have 
responded—confidentiality is rule number one—and 
as a client, I appreciate it. No harm done this time, 
though. What Jason failed to tell you is I’m on CCPA’s 
board and part of the project team. Before Jason 
stopped you, things were getting interesting. Please 
continue.” 


Sammye, getting a nod from Jason, shares, “We'll 
probably have to do some recoding of the age and race 
variables for the patterns to emerge clearly. The team is 
also interested in the differences between ethnic groups 
in future performance preferences. We’ ve also finished 
coding each patron’s address with its GPS (Geographic 
Positioning System) code. The preliminary mapping 
begins tomorrow; Jason hired a master’s candidate in 
geography to provide the mapping. I’ve scheduled a 
conference call for... (Sammye flips her desk calendar 
pages to the following week)... Friday of next week 
with Jackson Murray and other members of the CCPA 
project team.” 

“When the board approved your proposed analysis 
plan,” queries Myra, “I don’t remember seeing any 
reference to those boxlike diagrams with tails I see on 
that graph you just handed to Jason.” 

“Most of what the team will be doing the next three 
days,” intervenes Jason, “involves more graphical 
displays than statistical ones. Right now we’re just 
getting a sense of what the data are telling us. We’ll 
decide what, if any, new analyses to add to the proposed 
plan by this Friday. It’s this early work that lays the 
groundwork for the more sophisticated analyses that 
follow. There isn’t anything glamorous about it, but 
without it we might miss some crucial findings.” 

Jason pauses for effect and then says, “By the way, 
that ‘little diagram’ is called a boxplot. I actually did several 
during the preliminary analysis phase for MindWriter’s 
CompleteCare study. I didn’t give them to you because I 
would have had to explain how to interpret them and...” 

“...and anything you have to explain isn’t clear 
enough,” finishes Myra. 
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> Exploratory Data Analysis 


The convenience of data entry via spreadsheet, optimal mark recognition (OMR), or the data editor 
of a statistical program makes it tempting to move directly to statistical analysis. That temptation is 
even stronger when the data can be entered and viewed in real time. Why waste time finding out if the 
data confirm the hypothesis that motivated the study? Why not obtain descriptive statistical summaries 
(based on our discussion in Appendix 15a) and then test hypotheses? 

Exploratory data analysis is both a data analysis perspective and a set of techniques. In this chapter, 
we will present unique and conventional techniques including graphical and tabular devices to visual- 
ize the data. Exhibit 16-1 reminds you of the importance of data visualization as an integral element in 
the data analysis process and as a necessary step prior to hypothesis testing. In Chapter 3, we said re- 
search conducted scientifically is a puzzle-solving activity as well as an attitude of curiosity, suspicion, 
and imagination essential to discovery. It is natural, then, that exploration and examination of the data 
would be an integral part of our data analysis perspective. 

In exploratory data analysis (EDA) the researcher has the flexibility to respond to the patterns re- 
vealed in the preliminary analysis of the data. Thus, patterns in the collected data guide the data analy- 
sis or suggest revisions to the preliminary data analysis plan. This flexibility is an important attribute 
of this approach. When the researcher is attempting to prove causation, however, confirmatory data 
analysis is required. Confirmatory data analysis is an analytical process guided by classical statistical 
inference in its use of significance testing and confidence.! 


>Exhibit 16-1 Data Exploration, Examination, and Analysis in the Research Process 
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One authority has compared exploratory data analysis to the role of police detectives and other 
investigators and confirmatory analysis to that of judges and the judicial system. The former are in- 
volved in the search for clues and evidence; the latter are preoccupied with evaluating the strength of 
the evidence that is found. Exploratory data analysis is the first step in the search for evidence, without 
which confirmatory analysis has nothing to evaluate.’ Consistent with that analogy, EDA shares a com- 
monality with exploratory designs, not formalized ones. Because it doesn’t follow a rigid structure, it 
is free to take many paths in unraveling the mysteries in the data—to sift the unpredictable from the 
predictable. 

A major contribution of the exploratory approach lies in the emphasis on visual representations and 
graphical techniques over summary statistics. Summary statistics, as you will see momentarily, may 
obscure, conceal, or even misrepresent the underlying structure of the data. When numerical summa- 
ries are used exclusively and accepted without visual inspection, the selection of confirmatory models 
may be based on flawed assumptions.* For these reasons, data analysis should begin with visual inspec- 
tion. After that, it is not only possible but also desirable to cycle between exploratory and confirmatory 
approaches. 


Frequency Tables, Bar Charts, and Pie Charts* 


Several useful techniques for displaying data are not new to EDA. They are essential to any examina- 
tion of the data. For example, a frequency table is a simple device for arraying data. An example is 
presented in Exhibit 16-2. It arrays data by assigned numerical value, with columns for percent, valid 
percent (percent adjusted for missing data), and cumulative percent. This nominal variable describes 
the perceived desirable minimum age to be permitted to own a social networking account. The same 
data are presented in Exhibit 16-3 using a pie chart and a bar chart. The values and percentages are 
more readily understood in this graphic format. 

When the variable of interest is measured on an interval-ratio scale and is one with many potential 
values, these techniques are not particularly informative. Exhibit 16-4 (page 405) is a condensed fre- 
quency table of the average annual purchases of PrimeSell’s top 50 customers. Only two values, 59.9 
and 66, have a frequency greater than |. Thus, the primary contribution of a table for this data is an 
ordered list of values. If the table were converted to a bar chart, it would have 48 bars of equal length 
and two bars with two occurrences. Bar charts do not reserve spaces for values where no observations 
occur within the range. Constructing a pie chart for this variable would also be pointless. 


Exhibit 16-2 A Frequency Table (Minimum Age for Social Networking) 


Valid Cumulative 
Value Label Frequency Percent Percent Percent 
21 years old 1 60 6 6 6 
18 years old min 2 180 18 18 24 
16 years old min 3 330 33 33 ley 
13 years old min 4 280 28 28 85 
10 years old min 5) 50 5 ‘s) 90 
Any age 6 60 6 6 96 
No opinion 7 40 4 4 100 
1,000 100 100 


Valid Cases 1,000; Missing Cases 0 
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>Exhibit 16-3 Nominal Displays of Data (Minimum Age for Social Networking) 


Minimum Age for Social Networking 


Percent 
HH 21 years old 6 
18 years old 18 
i 16 years old 33 
i 13 years old 28 


[10 years old 5 
Any age 6 
No opinion 


Minimum Age for Social Networking 


Percent 


21 18 16 13 10 Any 
Age 


Histograms 


The histogram is a conventional solution for the display of interval-ratio data. Histograms are used 
when it is possible to group the variable’s values into intervals. Histograms are constructed with bars 
(or asterisks) that represent data values, where each value occupies an equal amount of area within the 
enclosed area. Data analysts find histograms useful for (1) displaying all intervals in a distribution, 
even those without observed values, and (2) examining the shape of the distribution for skewness, 
kurtosis, and the modal pattern. When looking at a histogram, one might ask: Is there a single hump 
(a mode)? Are subgroups identifiable when multiple modes are present? Are straggling data values 
detached from the central concentration?° 

The values for the average annual purchases variable presented in Exhibit 16-4 were measured on 
a ratio scale and are easily grouped. Other variables possessing an underlying order are similarly ap- 
propriate for histograms. A histogram would not be used for a nominal variable like minimum age for 
social networking (Exhibit 16-3) that has no order to its categories. 

A histogram of the average annual purchases is shown in Exhibit 16-5. The midpoint for each inter- 
val for the variable of interest, average annual purchases, is shown on the horizontal axis; the frequency 


>Exhibit 16-4 Average Annual Purchases of PrimeSell’s Top 50 Customers 
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Value Frequency 


75.6 1 
76.4 1 
Thd) 1 
78.9 1 
80.9 1 
82.2 1 
82.5 1 
86.4 1 
88.3 1 
102.5 1 
104.1 1 
110.4 1 
111.9 1 
118.6 1 
123.8 1 
131.2 1 
140.9 1 
146.2 1 
153.2 1 
163.2 1 
166.7 1 
183.2 {| 
206.9 1 
218.2 eile 
Total 50 


Percent 
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>Exhibit 16-5 Histogram of PrimeSell’s Top 50 Customers’ Average Annual Purchases 
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Novation’s Data Visualization Takes a Unique Approach 


Founded in 1998, Novation is a leading health care supply 
chain expertise and contracting company for more than 65,000 
members and affiliates. The health care organizations served by 
Novation face increasing pressures on their return on investment 
in the midst of rising costs, increasing uncompensated care, 
nursing shortages, and a national economic crisis. 

Melinda Gardner, vice president, strategic information for 
Novation, provides leadership, direction and strategic focus for 
activities related to Novation’s business intelligence activities. 
Her team is critical to the successful launch of Novation’s new 
data visualization product. “Product number and naming con- 
ventions aren’t standardized across the health care industry,” 
noted Gardner in a presentation on visual data discovery and 
self-service dashboards for business. That makes sharing infor- 
mation, especially with health-care systems about suppliers and 
purchasing behavior, problematic. With their data visualization 
tools, Novation wanted to “advance the way the organization 
thinks about data,” shared Gardner. Novation amassed data 
from member hospital purchase orders and invoices, sales data, 
distribution data and supplier and price management contracts, 
and made this data available to members not only on desktops, 
but on their iPhone or iPad. The new system needs to serve 
a variety of business users, including everyone from executive 
suite decision-makers to sales teams to finance. Novation did 
not want to limit users to the typical approach which usually 
consisted of an [T-developed report with data forced into the 
model. Rather, Novation wanted these users to be able to ‘play 
in the data,’ and have the ability to experiment with additional or 
alternative views. 

Novation partnered with MicroStrategy to implement this in- 
novative technology. The result is a product that enables busi- 
ness users to create ad hoc analysis with sophisticated data 
visualizations in a very quick way. Novation’s data visualization 
product is now launched and users are sharing best-practices 
reports—and doing this with little technical support. “Training 
was critical, but adoption has not been difficult,” shared Gardner. 
With Microstrategy’s help, Novation leveraged intelligent cubes 


Ineed information | Howcan!getreports 
now... how can | that include all the 
get quicker access? information | need? 


| want to 
create my own 
customized 
reports. 


Internal Customer 
Challenges To 


Data Visualization 
Deployment 


I have to go to 
3 different 


| want my 
information 
presented in a 
more insightful 
fashion. 


for this product, ensuring that a large amount of data and a 
large number of metrics could be accessed in a high perfor- 
mance environment. Complex analysis and visualization can be 
accomplished in a matter of minutes by use of the intelligent 
server. Engagement of the business users during the planning 
stages ensured early adoption of this product. Pre-populated 
analysis and use cases were utilized during training, and users 
were shown how to add or delete data fields and manipulate 
data charting to make data patterns—and outliers—come to life. 
Using these data visualization tools, Novation will continue to find 
ways to assist hospitals in maximizing the value of contracts and 
reducing supply chain costs. 

Novation serves the members and affiliates of VHA Inc. (a 
nationwide network of community-owned health care systems), 
UHC (a nationwide alliance representing approximately 90% of the 
nation’s non-profit academic medical centers), Children’s Hospi- 
tal Association (an alliance of the nation’s leading children’s hos- 
pitals), and Provista, LLC (a group purchasing organization that 
provides supply chain management and procurement services). 


www.novationco.com; www.microstrategy.com 


or number of observations in each interval, on the vertical axis. We erect a vertical bar above the mid- 
point of each interval on the horizontal scale. The height of the bar corresponds with the frequency of 
observations in the interval above which it is erected. This histogram was constructed with intervals 
20 increments wide, and the last interval contains only two observations, 206.9 and 218.2. These val- 
ues are found in PrimeSell’s average annual purchases frequency table (Exhibit 16-4). Intervals with 
0 counts show gaps in the data and alert the analyst to look for problems with spread. When the upper 
tail of the distribution is compared with the frequency table, we find three extreme values (183.2, 206.9, 
and 218.2). Along with the peaked midpoint and reduced number of observations in the upper tail, this 
histogram warns us of irregularities in the data. 
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Stem-and-Leaf Displays® 


The stem-and-leaf display is a technique that is closely related to the histogram. It shares some of 
the histogram’s features but offers several unique advantages. It is easy to construct by hand for small 
samples or may be produced by computer programs. In contrast to histograms, which lose informa- 
tion by grouping data values into intervals, the stem-and-leaf presents actual data values that can be 
inspected directly, without the use of enclosed bars or asterisks as the representation medium. This 
feature reveals the distribution of values within the interval and preserves their rank order for finding 
the median, quartiles, and other summary statistics. It also eases linking a specific observation back to 
the data file and to the subject that produced it. 

Visualization is the second advantage of stem-and-leaf displays. The range of values is apparent at a 
glance, and both shape and spread impressions are immediate. Patterns in the data—such as gaps where 
no values exist, areas where values are clustered, or outlying values that differ from the main body of 
the data—are easily observed. 

To develop a stem-and-leaf display for the data in Exhibit 16-4, the first digits of each data item 
are arranged to the left of a vertical line. Next, we pass through the average annual purchases per- 
centages in the order they were recorded and place the last digit for each item (the unit position, 
1.0) to the right of the vertical line. Note that the digit to the right of the decimal point is ignored. 
The last digit for each item is placed on the horizontal row corresponding to its first digit(s). Now it 
is a simple matter to rank-order the digits in each row, creating the stem-and-leaf display shown in 
Exhibit 16-6. 

Each line or row in this display is referred to as a stem, and each piece of information on the stem is 
called a leaf: The first line or row is 


5|455666788889 


The meaning attached to this line or row is that there are 12 items in the data set whose first digit is five: 
54, 55, 55, 56, 56, 56, 57, 58, 58, 58, 58, and 59. The second line, 


6|12466799 
shows that there are eight average annual purchase values whose first digit is six: 61, 62, 64, 66, 66, 
67, 69, and 69. 


When the stem-and-leaf display shown in Exhibit 16-6 is turned upright (rotated 90 degrees to the 
left), the shape is the same as that of the histogram shown in Exhibit 16-5. 


>Exhibit 16-6 A Stem-and-Leaf Display of PrimeSell’s Average Annual Purchases Data 
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closeup 


Because the researcher’s primary job is to discover the mes- The presence of any number within a table is for comparison 
sage revealed by the data, he or she needs every tool to reveal with a similar number—from last year, from another candidate, 
the message. Authors Sally Bigwood and Melissa Spore in their from another machine, against a goal, and so forth. Using the 
book Presenting Numbers, Tables, and Charts suggest that the author’s rules for table creation, a researcher exploring data by 
table is the ultimate tool for extracting knowledge from data. constructing a table should: 


e Round numbers. 


Arrange the num- 
bers to reveal 
patterns. 


e Use aver- 
ages, totals, or 
percentages to 
achieve focus. 


Compare like scales 
in a single table. 


Choose simplicity 
over complexity. 


Use empty space 
and design to guide 
the eye to numbers 
that must be com- 
pared and to make 
patterns and excep- 
tions stand out. 


e Summarize each 
data display. 


Label and title 
tables for clarity of 
message. 


Rounded numbers can be most easily compared, enabling us to more easily determine the ratio 
or relationship of one number to another. 


If precision is critical to the number (e.g., you are researching taxes or design specifications or 
drug interactions), don’t round the numbers. 


Order numbers from largest to smallest number. 
e In a vertically arranged table, order the largest number at the top. 
e In ahorizontal arrangement, order the largest numbers on the left. 


When looking for changes over time, order the numbers by year, from most distant (left or top) to 
most recent. 


An average provides a point for comparison. 
e Don’t use an average if the raw data reveal a bimodal distribution. 


Totals emphasize the big picture. 


Percentages show proportionate relationships more easily than raw data. 


Convert numbers to a common scale when the numbers reflect different scales (e.g., grams 
versus ounces of cereal consumption; monthly salary data versus hourly wage data). 


Several smaller tables reveal patterns better rather than one large, complex table. 


Complex tables are used as a convenient reference source for multiple elements of data. 


Design a table with a smaller number of columns than rows. 
Single-space numbers that must be compared. 


Use gridlines to group numbers within a table; avoid gridlines between numbers that must 
be compared. 


Use empty space to create gutters between numbers in simple tables. 


Right-align column headers and table numbers. 


Write a phrase or sentence that summarizes your interpretation of the data presented; don’t 
leave interpretation to chance. 

e Summary statements might be used as the title of a table or chart in the final research report. 
e The summary need not mention any numbers. 


Titles should be comprehensive: Include what (Subject of the title or message), where (if data 
have a geographic base), when (date or time period covered), and unit of measure. 


Include common information in the title: It lengthens a title but shortens the table’s column 
headings. 


Avoid abbreviations in column headings unless well Known by your audience. 


Avoid footnotes; if used, use symbols—like the asterisk—rather than numbers (numbers used as 
footnotes can be confused with the content numbers of the table). 


For reference, provide an undertable source line for later reference. 
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>closeupcont’d 


AN EXAMPLE 


Assume you were adetermining whether to expand into western Europe with distribution facilities to service online purchases of your 
specialty goods company. 


Table 1 Spending by Internet Users in Selected Western 
European Countries 2010 (EUROs in Billions) 


Annual Annual 

Spending Purchases 

Germany Euro 658.0 20 
Spain Euro 560.1 10 


We start with the above table that presents data developed from several studies on online shopping and purchasing behavior in 
selected countries in western Europe. The data are ordered alphabetically by country. While arranging in alphabetical order may be 
ideal for randomization or reduction of bias, it isn’t a logical choice for clarity of data presentation. 

What data might you need to help you make your decision about distribution facilities? Do you need to know the average 
transaction size? If you don’t know the conversion rate of the euro to the dollar, can you interpret the table? Should you put 
your investment in the United Kingdom or elsewhere? 


Table 2 E5 Per Capita One-Year Online Spending (201 0) 


Annual Spending Average Annual Annual Spending 

(EUROs) Purchases (USS) 

Germany 658.0 20 500.0 
Italy S455 14 262.6 


Currency Exchange Rate: 1 US$ = 1.316 EURO 


Table 2 recasts the data using Bigwood and Spore’s guidelines. First the table title has changed; now the annual period on which 
the spending data are based is more obvious, as well as the fact that we are looking at spending per capita for the top 5 European 
Union performers, known as the ES. We've also changed the column headers to reflect currency, and we have right-justified the 
headers and the numbers. We’ve rearranged the table by Average Spending (EURO) in descending order and interpreted the (EURO) 
column by adding a dollar conversion column. We might not need the rightmost column if we were euro spenders ourselves but, if we 
are more familiar with another currency, the addition of this column helps us interpret the data. With this arrangement, does Germany 
look attractive? While it might not currently appear to be as strong a contender as the United Kingdom, we know it is fiscally strong 
and located in a more central location to the other countries being considered. 
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>closeupcont’d 


Table 3 E5 Per Capita One-Year Online Spending (2010) 


Annual Spending Average Annual Annual 
(EURO) Purchases Spending (US$) 


Germany 658.0 


Ne) 
(=) 


500.00 Above Average Frequency Buyers 


Italy 345.5 


are 
iS 


262.57 Below Average Frequency Buyers 


Average 902.6 
Currency Exchange Rate: 1 US$ = 1.316 EURO 


ai 
© 
i) 


685.88 


Table 3 offers a simple addition: the average of the columns. And the table is now ordered by the Average Annual Purchases within 
the one-year period studied. This presentation allows the interpreter of the data to determine which countries are buying above the 
average for western Europe and which are buying below the average. Germany is looking pretty good. Right? 


Table 4 ES Per Capita One-Year Online Spending (2010) 


Annual Spending Annual Spending Average Annual Average Transaction Average Transaction 
(EUROs) (USS) Purchases (EURO) (US$) 


Spain 560.1 425.61 


= 
S) 
oO 
(o>) 
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42.56 


I) 
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Germany 658.0 500.00 25.00 


Average 902.6 685.88 
Currency Exchange Rate: 1 US$ = 1.316 EURO 
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35.72 


Table 4 offers a recasting of the data based on a newly calculated column, Average Transaction (Euro). Germany doesn’t look so 
attractive now. This is especially true if we are a seller of specialty merchandise with higher prices. 


Table 5 E5 Per Capita One-Year Online Spending (2010 vs. 2015) 


Estimated Average Estimated Average 
Estimated Percent Estimated Internet Transaction 2015 Transaction 2015 
Internet Users 2015 Users 2015 (millions) (EURO) (US$) 
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wo 
ol 
Co 
(o>) 


Italy 42.14 


ico} 
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United Kingdom 68.48 
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Average 42.14 
Unlike previous tables Table 5 is ordered from least to most based on the estimated 2015 Average Transaction and we’ve used pro- 
jections for 2015 purchase data. If you were blurry-eyed from studying numerous tables, you might quickly glance at the table, and 
think Germany as a distribution center should be the obvious choice. While Germany might still be a contender (given its stronger 
financial position in the European Union), its smaller projected transaction size might make it less so for a specialty goods distribu- 
tor. So a very important caution on tables: As a researcher you want to strive for consistency. If you are ordering from most to least, 
choose this arrangement for every table you do. 

After recasting the data in various tables, where would you put your distribution center? 


Sources: “Europe,” NewMedia TrendWatch, accessed August 1, 2012 (http://www.newmediatrendwatch.com/regional-overview/103-europe?showall=1). 
EURO to Dollar exchange rates, X-Rates, accessed July 28, 2012 (http://www.x-rates.com/table/?from=EUR). “Historical Exchange Rates: EURO 
to Dollar,” Oanda, accessed July 28, 2012 (http://www.oanda.com/currency/historical-rates/). Matt Creamer and Rupal Parekh, “Why the Euro Cri- 
sis Is Your Business Problem, Too,” Advertising Age, July 28, 2012, accessed July 28, 2012 (http://adage.com/article/news/euro-crisis-business- 
problem/236235/?utm_source=daily_email&utm_medium=newsletter&utm_campaign=adage). 
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>Exhibit 16-7 Pareto Diagram of MindWriter Repair Complaints 
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Pareto Diagrams 


Pareto diagrams derive their name from a 19th-century Italian economist. In quality management, 
J. M. Juran first applied this concept by noting that only a vital few defects account for most problems 
evaluated for quality and that the trivial may explain the rest. Historically, this has come to be known 
as the 80/20 rule—that is, an 80 percent improvement in quality or performance can be expected by 
eliminating 20 percent of the causes of unacceptable quality or performance. 

The Pareto diagram is a bar chart whose percentages sum to 100 percent. The data are derived 
from a multiple-choice, single-response scale; a multiple-choice, multiple-response scale; or frequency 
counts of words (or themes) from content analysis. The respondents’ answers are sorted in decreas- 
ing importance, with bar height in descending order from left to right. The pictorial array that results 
reveals the highest concentration of improvement potential in the fewest number of remedies. An 
analysis of MindWriter customer complaints is depicted as a Pareto diagram in Exhibit 16-7. The cu- 
mulative frequency line in this exhibit shows that the top two problems (the repair did not resolve the 
customer’s problem, and the product was returned multiple times for repair) accounted for 80 percent 
of the perceptions of inadequate repair service. 


Boxplots’ 


The boxplot, or box-and-whisker plot, is another technique used frequently in exploratory data analy- 
sis.’ A boxplot reduces the detail of the stem-and-leaf display and provides a different visual image 
of the distribution’s location, spread, shape, tail length, and outliers. Boxplots are extensions of the 
five-number summary of a distribution. This summary consists of the median, the upper and lower 
quartiles, and the largest and smallest observations. The median and quartiles are used because they 
are particularly resistant statistics. Resistance is a characteristic that “provides insensitivity to local- 
ized misbehavior in data.’” Resistant statistics are unaffected by outliers and change only slightly in 
response to the replacement of small portions of the data set. 

Recall the discussion of the mean and standard deviation in Appendix 15a. Now assume we take a 
data set [5,6,6,7,7,7,8,8,9] and calculate its mean. The mean of the set is 7; the standard deviation 1.22. 
If the 9 is replaced with 90, the mean becomes 16 and the standard deviation increases to 27.78. The 
mean is now two times larger than most of the numbers in the distribution, and the standard deviation 
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>Exhibit 16-8 Boxplot Components 
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is more than 22 times its original size. Changing only one of nine values has disturbed the location and 
spread summaries to the point where they no longer represent the other eight values. Both the mean and 
the standard deviation are considered nonresistant statistics; they are susceptible to the effects of ex- 
treme values in the tails of the distribution and do not represent typical values well under conditions of 
asymmetry. The standard deviation is particularly problematic because it is computed from the squared 
deviations from the mean.’ In contrast, the median and quartiles are highly resistant to change. When 
we changed the 9 to 90, the median remained at 7 and the lower and upper quartiles stayed at 6 and 8, 
respectively. Because of the nature of quartiles, up to 25 percent of the data can be made extreme with- 
out perturbing the median, the rectangular composition of the plot, or the quartiles themselves. These 
characteristics of resistance are incorporated into the construction of boxplots. 

Boxplots may be constructed easily by hand or by computer programs. The basic ingredients of the 
plot are: 


1. The rectangular plot that encompasses 50 percent of the data values. 
2. A center line (or other notation) marking the median and going through the width of the box. 
3. The edges of the box, called hinges. 


4. The “whiskers” that extend from the right and left hinges to the largest and smallest values.!! 


These values may be found within 1.5 times the interquartile range (IQR) from either edge of the 
box. These components and their relationships are shown in Exhibit 16-8. 

When you are examining data, it is important to separate legitimate outliers from errors in measure- 
ment, editing, coding, and data entry. Outliers, data points that exceed + 1.5 the interquartile range, 
reflect unusual cases and are an important source of information for the study. They are displayed or 
given special statistical treatment, or other portions of the data set are sometimes shielded from their 
effects. Outliers that are entry mistakes should be corrected or removed during editing. 

Exhibit 16-9 summarizes several comparisons that are of help to the analyst. Boxplots are an 
excellent diagnostic tool, especially when graphed on the same scale. The upper two plots in the ex- 
hibit are both symmetric, but one is larger than the other. Larger box widths are sometimes used when 
the second variable, from the same measurement scale, comes from a larger sample size. The box 
widths should be proportional to the square root of the sample size, but not all plotting programs ac- 
count for this.!* Right- and left-skewed distributions and those with reduced spread are also presented 
clearly in the plot comparison. Finally, groups may be compared by means of multiple plots. One 


>Exhibit 16-9 Diagnostics with Boxplots 
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Symmetric 


Symmetric— 
larger relative size in 
proportion to sample size 


Right skewed 


Left skewed 


Small spread 


Notched at the median for a test of 
the equality of population medians 


variation, in which a notch at the median marks off a confidence interval to test the equality of group 
medians, takes us a step closer to hypothesis testing.'? Here the sides of the box return to full width at 
the upper and lower confidence intervals. When the intervals do not overlap, we can be confident, at a 
specified confidence level, that the medians of the two populations are different. 

In Exhibit 16-10, multiple boxplots compare five sectors of PrimeSell’s customers by their average 
annual purchases data. The overall impression is one of potential problems for the analyst: unequal 


>Exhibit 16-10 Boxplot Comparison of Customer Sectors 
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variances, skewness, and extreme outliers. Note the similarities of the profiles of finance and retailing 
in contrast to the high-tech and insurance sectors. If hypothesis tests are planned, further examination 
of this plot for each sector would require a stem-and-leaf display and a five-number summary. From 
this, we could make decisions on the types of tests to select for confirmatory analysis (see Chapters 17, 
18, and the Online Learning Center supplement Multivariate Analysis, an Overview). 


Mapping 


Increasingly, participant data are being attached to their geographic dimension as Geographic Infor- 
mation System (GIS) software and coordinate measuring devices have become more affordable and 
easier to use. Essentially a GIS works by linking data sets to each other with at least one common data 
field (e.g., a household’s street address). The GIS allows the researcher to connect target and classifica- 
tion variables from a survey to specific geographic-based databases like U.S. Census data, to develop 
a richer understanding of the sample’s attitudes and behavior. When radio frequency identification 
(RFID) data become more prevalent, much behavioral data will be able to connect with these new 
geographically rich databases. 

The most common way to display such data is with a map. Colors and patterns denoting knowledge, 
attitude, behavior, or demographic data arrays are superimposed over street maps (finest-level GIS), 
block-group maps, or county, state, or country maps to help identify the best locations for stores based 
on demographic, psychographic, and life-stage segmentation data. Florists array promotional response 
information geographically and use the map to plan targeted promotions. Consumer and business-to- 
business researchers use mapping of data on ownership, usage level, and price sensitivity in plotting 
geographic rollouts of new products. Although this is an attractive option for exploratory analysis, it 
does take specialized software and hardware, as well as the expertise to operate it. Students are encour- 
aged to take specialized courses on GIS to expand their skill set in this growing area. 


Foreclosure Actions to Housing Units 
1 im 117 Housing Units 1 in 63,583 Housing Units 
i 


Sometimes there is no better way to display data than with a map. While in 2009, home foreclosures in the United 
States hit an all-time high, by 2012 some markets were beginning to rebound. At a quick glance, when geospatial data 
are mapped, you can tell which states were on the rebound. This map from RealtyTrac shows home foreclosure list- 
ings, by state for May 2012. 

www.realtytrac.com/trendcenter 
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Digital Natives Shift Attention Nearly Twice as Much 
as Their Digital Immigrant Counterparts 


A study commissioned by Time Warner’s Time Inc. and con- 
ducted by Boston’s Innerscope Research provides additional 
support that we are different than prior generations, especially 
when it comes to media consumption—the time spent with 
media, the attention paid to media messages and across media 
platforms, as well as the emotional consequences of these 
behavior patterns. 

Thirty subjects, half of whom were digital natives (having 
grown up with digital media) and half of whom were digital im- 
migrants (who came to digital in adulthood but grew up with 
traditional media), were monitored as they interacted with media 
throughout their normal day—amounting to a total of 300 hours 
of real time data. Each person wore glasses with an embed- 
ded camera to monitor what media platform (TV, computer, 
tablet, smart phone) he or she was using and the content at 
the time. Innerscope’s Biometric Monitoring System monitored 


time. The system provided quantifiable emotional engagement 
measurements. The study was designed to understand the 
role that media play in real lives and in real ways. Digital na- 
tives had much shorter attention spans, switching their focus on 
average 27 times per hour, while digital immigrants switched just 
17 times per hour. 

“The study’s findings mean the [creative] bar is set higher for 
advertisers,” shared Innerscope Research CEO Dr. Carl Marci. 
“ld be most alarmed about the challenging act of capturing 
the attention and emotional response of my target audience, 
because it’s almost like going from shooting fish in a barrel to 
[shooting] little minnows. The target has become faster, and the 
window of opportunity for capturing them has become smaller.” 

What type of study was conducted here? What design issues 
are addressed in this study? What concepts and constructs 
might be measured? 
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autonomic nervous system responses with a noninvasive belt 
worn around the lower ribcage of the participant during nonwork 


www.timewarner.com; www.innerscope.com 


Throughout this section we have exploited the visual techniques of exploratory data analysis to look 
beyond numerical summaries and gain insight into the patterns of the data. Few of the approaches have 
stressed the need for advanced mathematics, and all have an intuitive appeal for the analyst. When the 
more common ways of summarizing location, spread, and shape have conveyed an inadequate picture 
of the data, we have used more resistant statistics to protect us from the effects of extreme scores and 
occasional errors. We have also emphasized the value of transforming the original scale of the data 
during preliminary analysis rather than at the point of hypothesis testing. 


> Cross-Tabulation 


Depending on the management question, we can gain valuable insights by examining the data with 
cross-tabulation. Cross-tabulation is a technique for comparing data from two or more categorical 
variables such as gender and selection by one’s company for an overseas assignment. Cross-tabulation 
is used with demographic variables and the study’s target variables (operationalized measurement ques- 
tions). The technique uses tables having rows and columns that correspond to the levels or code values 
of each variable’s categories. Exhibit 16-11 is an example of a computer-generated cross-tabulation. 
This table has two rows for gender and two columns for assignment selection. The combination of the 
variables with their values produces four cells. Each cell contains a count of the cases of the joint clas- 
sification and also the row, column, and total percentages. The number of row cells and column cells is 
often used to designate the size of the table, as in this 2 X 2 table. The cells are individually identified 
by their row and column numbers, as illustrated. Row and column totals, called marginals, appear at 
the bottom and right “margins” of the table. They show the counts and percentages of the separate rows 
and columns. 

Cross-tabulation is a first step for identifying relationships between variables. When tables are 
constructed for statistical testing, we call them contingency tables, and the test determines if the 
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>Exhibit 16-11 SPSS Cross-Tabulation of Gender by Overseas Assignment Opportunity 


OVERSEAS ASSIGNMENT 

Cell 
content 

GENDER 

Male 

Marginals 

Cell 2, 1 
(row 2, column 1) 


classification variables are independent of each other (see chi-square in Chapter 17). Of course, tables 
may be larger than 2 X 2. 


The Use of Percentages 


Percentages serve two purposes in data presentation. First, they simplify the data by reducing all num- 
bers to a range from 0 to 100. Second, they translate the data into standard form, with a base of 100, for 
relative comparisons. In a sampling situation, the number of cases that fall into a category is meaning- 
less unless it is related to some base. A count of 28 overseas assignees has little meaning unless we 
know it is from a sample of 100. Using the latter as a base, we conclude that 28 percent of this study’s 
sample has an overseas assignment. 

Although the above is useful, it is even more useful when the research problem calls for a compari- 
son of several distributions of data. Assume the previously reported data were collected five years ago 
and the present study had a sample of 1,500, of which 360 were selected for overseas assignments. By 
using percentages, we can see the relative relationships and shifts in the data (see Exhibit 16-12). 


>Exhibit 16-12 Comparison of Percentages in Cross-Tabulation Studies by 
Overseas Assignment 


Study 1 Study 2 
OVERSEAS ASSIGNMENT OVERSEAS ASSIGNMENT 
Count Count 
Yes No Row Pct Yes No 
Col Pct | Row Col Pct Row 
Tot Pet 1 2 Total Tot Pct 1 2 Total 
GENDER 22 40 be GENDER 225 b75 400 
Male 1 35.5 b4.5 b2.0 Male 1 25.0 75.0 bO.0 
be.S 54.2 
15.0 45.0 
135 4BS bO0 
Female 2 T7025 40.0 
Bias 40.8 
Gist 31.0 
Column 3b0 1140 1500 


Total 24.0 76.0 100.0 
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Empowering Excel 


When developing data displays, some research analysts turn not 
to heavy duty statistical software packages like SPSS or SAS, 
but to the spreadsheet on their desktop. Two enterprising pro- 
fessors from Pennsylvania State University, Gary L. Lilien and 
Arvind Rangaswamy, have teamed to developed Microsoft Excel 
plug-ins that empower the spreadsheet to create some com- 
monly desired displays with just a few mouse clicks. “With this 
software product, users will be able to use the power of world- 
class analytics from within Excel, an interface with which they are 
already comfortable,” claims their website. 


of creating the display, such as the GE Matrix shown here. The 
user can practice the techniques on sample data from real com- 
panies or enter his or her own data. 

The plug-ins currently facilitate displays for forecasting, con- 
joint analysis, customer choice analysis, customer lifetime value 
analysis, GE Portfolio Matrix plotting, positioning analysis, re- 
source allocation analysis, new-product and service design, and 
segmentation/targeting analysis—all common analytical exer- 
cises for managers involved in marketing engineering. Thanks to 
Lilien and Rangaswamy, some sophisticated display techniques 
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Each plug-in offers a unique template for data entry where 
the charting directions are embedded. Once the data are en- 
tered, a series of windows guides the user through the process 


have been significantly simplified. 
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With two-dimension tables, the selection of a row or column will accentuate a particular distribution 
or comparison. This raises the question about which direction the percentages should be calculated. 
Most computer programs offer options for presenting percentages in both directions and interchanging 
the rows and columns of the table. But in situations in which one variable is hypothesized to be the pre- 
sumed cause, is thought to affect or predict a response, or is simply antecedent to the other variable, we 
label it the independent variable. Percentages should then be computed in the direction of this variable. 
Thus, if the independent variable is placed on the row, select row percentages; if it is on the column, 
select column percentages. In which direction should the percentages run in the previous example? If 
only the column percentages are reported, we imply that assignment status has some effect on gender. 
This is implausible. When percentages are reported by rows, the implication is that gender influences 
selection for overseas assignments. 

Care should be taken in interpreting percentages from tables. Consider again the data in 
Exhibit 16-12. From the first to the second study, it is apparent that the percentage of females selected 
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Internet-Age Researchers: Building Critical Transferable Skills 


According to New York Times columnist Steve Lohr, in the digital 
age statisticians “are changing the image of the profession as 
a place for dronish number-nerds. They are finding themselves 
increasingly in demand—and even cool.” Lohr asserts that the 
rapid ascendancy of statisticians, who can earn $125,000 in 
their first year after getting a doctorate, is the result of the re- 
cent explosion of digital data. With Web-based data expanding 
rapidly, up to fivefold by 2012, there are myriad opportunities for 
exploration and problem solving. 

Google’s chief economist, Hal Varian, explains the im- 
portance of the Internet’s free and ubiquitous data this way: 
“The ability to take data—to be able to understand it, to pro- 
cess it, to extract value from it, to visualize it, to communi- 
cate it—that’s going to be a hugely important skill in the next 
decades....” 

While statisticians are in high demand, Varian emphasizes 
the need for managers to understand data themselves. In old 
organizations, you had an “army of people digesting data and 
feeding it to decision makers at the top.” Today, it is essential 
that people can access, understand, and communicate insights 
from data analysis that affect everyday decisions. Using statisti- 
cal models, multivariate analysis, and data mining, Internet-age 


statisticians operate as “bridge scientists” engaged in the quest 
to find meaningful patterns in information while they advance 
business opportunities and identify risks. 

So where are the newly fashionable “number-nerds” going? 
Even in a bad economy, Wall Street, finance, pharmaceuticals, 
insurance, research labs, and the government are hiring. Many 
of those being hired are part of multi-billion-dollar acquisitions. 
Keeping pace with the expanding market for “business intelli- 
gence” software, SPSS offers software and data tools designed 
to help countless companies understand their consumers. 
SPSS was acquired by IBM for $1.2 billion. IBM also offered 
nearly $5 billion for the purchase of Cognos. Additionally, Oracle 
purchased Hyperion Solutions for roughly $3.3 billion, while 
SAP purchased Business Objects for $4.8 billion. And Micro- 
soft bought Farecast.com for $115 million to support its new 
venture, Bing Travel, that reveals if you should purchase an air- 
line ticket now or wait for a time closer to your desired flight’s 
departure. 

So, dive into data analysis and learn as much as you can. It’s 
a skill that is much in demand. 


www.google.com; www.SAP.com; www.spss.com 


for overseas assignments rose from 15.8 to 22.5 percent of their respective samples. This should not 
be confused with the percentage within each sample who were women with overseas assignments, 
a number which increased from 6 percent (Study 1) to 9 percent (Study 2). Among all overseas 
selectees, in the first study 21.4 percent were women, while in the second study, 37.5 percent were 
women. Similar comparisons can be made for the other categories. The tables verify an increase in 
women with overseas assignments, but we cannot conclude that their gender had anything to do with 
the increase. 

Percentages are used by virtually everyone dealing with numbers—but often incorrectly. The fol- 
lowing guidelines, if used during analysis, will help to prevent errors in reporting:'4 


e Averaging percentages. Percentages cannot be averaged unless each is weighted by the size of 
the group from which it is derived. Thus, a simple average will not suffice; it is necessary to use 
a weighted average. 


° Use of too large percentages. This often defeats the purpose of percentages—which is to sim- 
plify. A large percentage is difficult to understand and is confusing. If a 1,000 percent increase 
is experienced, it is better to describe this as a 10-fold increase. 


e Using too small a base. Percentages hide the base from which they have been computed. A fig- 
ure of 60 percent when contrasted with 30 percent would appear to suggest a sizable difference. 
Yet if there are only three cases in the one category and six in the other, the differences would 
not be as significant as they have been made to appear with percentages. 


e Percentage decreases can never exceed 100 percent. This is obvious, but this type of mistake 
occurs frequently. The higher figure should always be used as the base or denominator. For 
example, if a price was reduced from $1 to $.25, the decrease would be 75 percent (75/100). 
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>Exhibit 16-13 SPSS Cross-Tabulation with Control and Nested Variables 


Control Variable 


Category 1 Category 2 


Nested Variable Nested Variable 
cat l | cat 2 | cat 3 cat lh | cat 2 | cat 3 


Scuberer GWNS 5 5 4 


SEX OF EMPLOYEE 


MALES FEMALES 


MINORITY MINORITY 
CLASSIFICATION CLASSIFICATION 


NONWHITE NONWHITE 


EMPLOYMENT CATEGORY 


CLERICAL 

OFFICE TRAINEE 
SECURITY OFFICER 
COLLEGE TRAINEE 
EXEMPT EMPLOYEE 
MBA TRAINEE 
TECHNICAL 


Other Table-Based Analysis 


The recognition of a meaningful relationship between variables generally signals a need for further 
investigation. Even if one finds a statistically significant relationship, the questions of why and under 
what conditions remain. The introduction of a control variable to interpret the relationship is often 
necessary. Cross-tabulation tables serve as the framework. 

Statistical packages like Minitab, SAS, and SPSS have among their modules many options for the 
construction of n-way tables with provision for multiple control variables. Suppose you are interested 
in creating a cross-tabulation of two variables with one control. Whatever the number of values in the 
primary variables, the control variable with five values determines the number of tables. For some ap- 
plications, it is appropriate to have five separate tables; for others, it might be preferable to have adjoin- 
ing tables or have the values of all the variables in one. Management reports are of the latter variety. 
Exhibit 16-13 presents an example in which all three variables are handled under the same banner. 
Programs such as this one can handle far more complex tables and statistical information.’ 

An advanced variation on n-way tables is automatic interaction detection (AID). AID is a com- 
puterized statistical process that requires that the researcher identify a dependent variable and a set of 
predictors or independent variables. The computer then searches among up to 300 variables for the 
best single division of the data according to each predictor variable, chooses one, and splits the sample 
using a statistical test to verify the appropriateness of this choice. 

Exhibit 16-14 shows the tree diagram that resulted from an AID study of customer satisfaction with 
MindWriter’s CompleteCare repair service. The initial dependent variable is the overall impression of 
the repair service. This variable was measured on an interval scale of | to 5. The variables that contrib- 
ute to perceptions of repair effectiveness were also measured on the same scale but were rescaled to 
ordinal data for this example (1-2 = poor, 3 = average, and 4—5 = excellent). The top box shows that 
62 percent of the respondents rated the repair service as excellent (41% + 21%). The best predictor of 
repair effectiveness is “resolution of the problem.” 

On the left side of the tree, customers who rated “resolution of the problem” as poor have fewer ex- 
pectations being met or exceeded than the average for the sample (6 percent versus 62 percent). A poor 
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>Exhibit 16-14 Automatic Interaction Detection Example (MindWriter’s Repair Satisfaction) 


1 


Overall CompleteCare Impression 
(Expectations = number scale) 


Resolution of the problem 


Poor 
44% Met few 
35% Met some 
15% Met most 
6% Met all 
0% Exceeded 
noe 


Condition on arrival 


Poor 


63% Met few 
26% Met some 


9% Met most 

2% Met all 

0% Exceeded 
25) 


Service rep technical competence 


Speed of repair 


rating on “condition on arrival” exacerbates this, reducing the total satisfied group to 2 percent. From 
this example you can see that the researcher separately studied (applied AID to) each subgroup to find 
the variable that when split again makes the next largest contribution to understanding the consumers’ 
evaluation process—and to the reduction of unexplained variation in each subsample. This analysis 
alerts decision makers at MindWriter to the best- and worst-case scenarios for the CompleteCare ser- 
vice, how to recover during a problematic month, and which “key drivers,” or independent variables 
influencing the process, should receive corrective resources. 


Exploratory data analysis (EDA) provides a perspective and 
set of tools to search for clues and patterns in the data. 
EDA augments rather than supplants traditional statistics. 
In addition to numerical summaries of location, spread, 
and shape, EDA uses visual displays to provide a com- 
plete and accurate impression of distributions and variable 
relationships. 

Frequency tables array data from lowest to highest val- 
ues with counts and percentages. They are most useful for 
inspecting the range of responses and their repeated occur- 
rence. Bar charts and pie charts are appropriate for relative 
comparisons of nominal data. Histograms are optimally 


used with continuous variables where intervals group the re- 
sponses. The Pareto diagram is a bar chart whose percent- 
ages sum to 100 percent. The causes of the problem under 
investigation are sorted in decreasing importance, with bar 
height descending from left to right. Stem-and-leaf displays 
and boxplots are EDA techniques that provide visual repre- 
sentations of distributions. The former present actual data 
values using a histogram-type device that allows inspec- 
tion of spread and shape. Boxplots use the five-number 
summary to convey a detailed picture of a distribution’s 
main body, tails, and outliers. Both stem-and-leaf displays 
and boxplots rely on resistant statistics to overcome the 
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limitations of descriptive measures that are subject to ex- 
treme scores. 


percentages. The tabular structure is the framework for later 
statistical testing. Computer software for cross-classification 
analysis makes table-based analysis with one or more con- 
trol variables an efficient tool for data visualization and later 
decision making. An advanced variation on n-way tables is 
automatic interaction detection (AID). 


2 The examination of relationships involving categorical 
variables employs cross-tabulation. The tables used for 
this purpose consist of cells and marginals. The cells may 
contain combinations of count, row, column, and total 


paeviorns 


automatic interaction 
detection (AID) 423 


cross-tabulation 419 marginals 419 


exploratory data analysis (EDA) 406 nonresistant statistics 416 
boxplot 415 five-number summary 415 outliers 416 
cell 419 frequency table 407 Pareto diagram 415 
confirmatory data analysis 406 histogram 408 resistant statistics 415 
contingency table 419 interquartile range (IQR) 416 stem-and-leaf display 411 


control variable 423 


>dil ion lon 
Terms in Revie’ 
1 Defi i . Home Near Home Far 
SUNG Ol EXP Ialls Aid Receiving Aid —_- Receiving Aid 
a Marginals. 
Yes No Yes No Yes No 
b Pareto diagram. (%)  (%) (%) (%) (%) (%) 
c Nonresistant statistics. Drop Out 25 20 5 15 30 40 
d_ Lower control limit. Stay 75 80 95 85 70 60 


e The five-number summary. 


Making Research Decisions 

2 Suppose you were preparing two-way tables of percent- 
ages for the following pairs of variables. How would you run 
the percentages? 


4 Alocal health agency is experimenting with two appeal letters, 
A and B, with which to raise funds. It sends out 400 of the A 
appeal and 400 of the B appeal (each subsample is divided 
equally among working-class and middle-class neighborhoods). 


a Age and consumption of breakfast cereal. The agency secures the results shown in the following table. 


b Family income and confidence about the family’s future. 
c Marital status and sports participation. 
d Crime rate and unemployment rate. 


You study the attrition of entering college freshmen 
(those students who enter college as freshmen but don’t 


a Which appeal is the best? 
b Which class responded better to which letter? 


c ls appeal or social class a more powerful independent 
variable? 


stay to graduate). You find the following relationships Appeal APPSE 
between attrition, aid, and distance of home from college. Middle Working Middle = Working 
What is your interpretation? Consider all variables and Class —_ Class Class Class 
relationships. (%) (%) (%) (%) 
Contribution 20 40 15 30 
No Contribution 80 60 85 70 


100 100 100 100 
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5 Assume you have collected data on sales associates of a 
large retail organization in a major metropolitan area. You 
analyze the data by type of work classification, education 
level, and whether the workers were raised in a rural or 
urban setting. The results are shown here. How would you 
interpret them? 


Annual Retail Employee Turnover per 100 Employees 


High Education Low Education 


Hourly Hourly Hourly 

Salaried Wage Salaried Wage Salaried Wage 
Rural 8 16 6 14 18 18 
Urban 12 16 10 12 19 20 


6 Asustek, the Taiwanese manufacturer that basically invented 
the netbook category, has been researching more radical 
design ideas, including a classy wrist-top computer, the 
Waveface Ultra. It is made from a bendable display that can 
connect to the Internet, make phone calls, and crunch data. 
Essentially, it’s a bracelet that acts like a smartphone. 


a How might you use such a device to display stimuli for 
respondents? 


b What is the interactive data exchange potential for 
researchers? 


Bringing Research to Life 

7 Identify the variables being cross-tabulated by Sammye. 
Identify some plausible reasons why such an exploration 
would be a good idea. 


From Concept to Practice 
8 Use the data in Exhibit 16-5 to construct a stem-and-leaf 
display. 


a Where do you find the main body of the distribution? 


b How many values reside outside the inner fence(s)? 


From the Headlines 

9 An article posted by AdAge Media News on June 18, 2012 
revealed that “a study commissioned by the Online Publish- 
ers Association found that 54 percent of tablet users prefer 
free, ad-supported apps over paid ones, up from 40 per- 
cent a year ago. At the same time, just 19 percent of users 
say they prefer to pay more for apps with no ads, down 
from 30 percent last year.” 


a Which charting methods offer an accurate reflection of 
the data? 


b Which provides the most meaning? Why? 


rT 


Agri Comp 


Mastering Teacher Leadership 


NCRCC: Teeing Up and New Strategic Direction 


Proofpoint: Capitalizing on a Reporter’s Love of Statistics 


*You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available from 


the Online Learning Center. 


chapter 17 


learningobjectives 
After reading this chapter, you should understand . . . 


The nature and logic of hypothesis testing. 

What a statistically significant difference is. 

The six-step hypothesis testing procedure. 

The differences between parametric and nonparametric tests and when to use each. 
The factors that influence the selection of an appropriate test of statistical significance. 


How to interpret the various test statistics. 


6 GA fact is a simple statement that everyone 
believes. It is innocent, unless found guilty. 
A hypothesis is a novel suggestion that no 
one wants to believe. It is guilty, until found 


effective. ? 9 


Edward Teller, theoretical physicist, 
known colloquially as “the father of the hydrogen bomb” 
(1908-2003) 


bringingresearcnhtolife 


“Sara, 'd like to meet with you about verifying the 
gender and age differences on the alcohol issue for 
Center City for Performing Arts Association.” Jason 
makes his way through the cluttered outer office, 
stepping around piles of printouts, topped with 
sketched graphs or detailed cross-tabulated tables with 
handwritten notes. 

Moments later, Sara enters, carrying the cross- 
tabulated data to which Jason had referred. 

Jason smiles, waiting for her to settle. “So what 
have you got?” 

“There is definitely a difference in attitude about 
serving alcohol during intermission at performances. 
But it doesn’t appear to be quite what the CCPA board 
expected.” 

“How so?” 

“Well, the younger patrons seem somewhat divided, 
while those between 35 and 54 are against and those 55 
and over are strongly in favor.” 

“What was your original hypothesis?” 

“Based on your meeting notes from the project session 
with the CCPA board, I formulated the hypothesis that 
there would be a difference on the alcohol issue based 


on age,” shares Sara. “But I assumed that the younger 
the patrons, the more in favor of alcohol they would be. 
The numbers just aren’t supporting that. And I’m not so 
sure that age is the right variable to look at.” 

Jason extends his hand across his desk. “Let me see 
the statistics on age.” 

“And I’ve got the stats on gender too,” offers Sara. 

“Are those in line with your hypothesis?” 

“Not really,” shares Sara. “Men and women are all 
over the place. I hypothesized that men would be in 
favor while women would be against. That’s just not 
panning out.” 

Jason glances at the printout, pleased to see that 
her interpretation of the statistics is correct. “Looks 
like you have some work yet, to determine the pockets 
of resistance. Since the sample split—wasn’t it 
57 percent in favor to 43 percent against?—we don’t 
want to recommend that CCPA proceed without being 
able to tell the board the likely direction of potential 
trouble. 

“Sometimes our preliminary analysis plan can take 
us only so far,” comments Jason. “Let’s talk about 
what tests you plan to run now.” 
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> Introduction 


In Chapters 15 and 16, we discussed the procedures for data preparation and preliminary analysis. The 
next step for many studies is hypothesis testing. 

Just as your understanding of scientific reasoning was important in the last two chapters, recollec- 
tion of the specific differences between induction and deduction is fundamental to hypothesis testing. 
Inductive reasoning moves from specific facts to general, but tentative, conclusions. We can never be 
absolutely sure that inductive conclusions are flawless. With the aid of probability estimates, we can 
qualify our results and state the degree of confidence we have in them. Statistical inference is an appli- 
cation of inductive reasoning. It allows us to reason from evidence found in the sample to conclusions 
we wish to make about the population. 

Inferential statistics is the second of two major categories of statistical procedures, the other being 
descriptive statistics. We used descriptive statistics in Chapter 14 when describing distributions. Under 
the heading inferential statistics, two topics are discussed in this book. The first, estimation of popula- 
tion values, was used with sampling in Chapter 14, but we will return to it here briefly. The second, 
testing statistical hypotheses, is the primary subject of this chapter. There are more examples of hy- 
pothesis tests in this chapter than most students will need for a term project or early assignments in 
their research careers. A section on nonparametric techniques in Appendix C provides further study for 
readers with a special interest in nominal and ordinal variables. 

After you have detailed your hypotheses in your preliminary analysis planning, the purpose of hy- 
pothesis testing is to determine the accuracy of your hypotheses due to the fact that you have collected a 
sample of data, not a census. Exhibit 17-1 reminds you of the relationships among your design strategy, 
data collection activities, preliminary analysis, and hypothesis testing. 

We evaluate the accuracy of hypotheses by determining the statistical likelihood that the data reveal 
true differences—not random sampling error. We evaluate the importance of a statistically significant 
difference by weighing the practical significance of any change that we measure. 

Although there are two approaches to hypothesis testing, the more established is the classical or 
sampling-theory approach. Classical statistics are found in all of the major statistics books and are 
widely used in research applications. This approach represents an objective view of probability in 
which the decision making rests totally on an analysis of available sampling data. A hypothesis is es- 
tablished; it is rejected or fails to be rejected, based on the sample data collected. 

The second approach is known as Bayesian statistics, which are an extension of the classical ap- 
proach. They also use sampling data, but they go beyond to consider all other available information. 
This additional information consists of subjective probability estimates stated in terms of degrees of be- 
lief. These subjective estimates are based on general experience rather than on specific collected data. 
Various decision rules are established, cost and other estimates can be introduced, and the expected 
outcomes of combinations of these elements are used to judge decision alternatives. 


Statistical Significance 


Following classical statistics approach, we accept or reject a hypothesis on the basis of sampling 
information alone. Since any sample will almost surely vary somewhat from its population, we must 
judge whether the differences are statistically significant or insignificant. A difference has statistical 
significance if there is good reason to believe the difference does not represent random sampling 
fluctuations only. For example, Honda, Toyota, Chrysler, Nissan, Ford, and other auto companies pro- 
duce hybrid vehicles using an advanced technology that combines a small gas engine with an electric 
motor. The vehicles run on an electric motor at slow speeds but shift to both the gasoline motor and the 
electric motor at city and higher freeway speeds. Their advertising strategies focus on fuel economy. 
Let’s say that the hybrid Toyota has maintained an average of about 60 miles per gallon (mpg) with a 
standard deviation of 10 mpg. Suppose researchers discover by analyzing all production vehicles that 
the mpg is now 61. Is this difference statistically significant from 60? Of course it is, because the dif- 
ference is based on a census of the vehicles and there is no sampling involved. It has been demonstrated 
conclusively that the population average has moved from 60 to 61 mpg. Although it is of statistical 
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>Exhibit 17-1 Hypothesis Testing and the Research Process 


Formulate Preliminary 
Hypotheses 


Exploration Exploration 


' Research | 
. Proposal | 


Research Design 


Data Collection Sampling 
Design Design 


Question & Instrument 
Pilot Testing 


| Refine Hypotheses 
Data Visualization 


Data Collection & 
Preparation 


Research Reporting 


Management 
Decision 


significance, whether it is of practical significance is another question. If a decision maker judges that 
this variation has no real importance, then it is of little practical significance. 

Since it would be too expensive to analyze all of a manufacturer’s vehicles frequently, we resort to 
sampling. Assume a sample of 25 cars is randomly selected and the average mpg is calculated to be 
64. Is this statistically significant? The answer is not obvious. It is significant if there is good reason to 
believe the average mpg of the total population has moved up from 60. Since the evidence consists of 
only a sample, consider the second possibility: that this is only a random sampling error and thus is not 
significant. The task is to decide whether such a result from this sample is or is not statistically signifi- 
cant. To answer this question, one needs to consider further the logic of hypothesis testing. 
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The Logic of Hypothesis Testing 


In classical tests of significance, two kinds of hypotheses are used. The null hypothesis (H,) is used for 
testing. It is a statement that no difference exists between the parameter (a measure taken by a census of 
the population or a prior measurement of a sample of the population) and the statistic being compared 
to it (a measure from a recently drawn sample of the population). Analysts usually test to determine 
whether there has been no change in the population of interest or whether a real difference exists. Why 
not state the hypothesis in a positive form? Why not state that any difference between the sample 
statistic and the population parameter is due to some reason? Unfortunately, this type of hypothesis 
cannot be tested definitively. Evidence that is consistent with a hypothesis stated in a positive form can 
almost never be taken as conclusive grounds for accepting the hypothesis. A finding that is consistent 
with this type of hypothesis might be consistent with other hypotheses too, and thus it does not demon- 
strate the truth of the given hypothesis. 

For example, suppose a coin is suspected of being biased in favor of heads. The coin is flipped 100 
times and the outcome is 52 heads. It would not be correct to jump to the conclusion that the coin is 
biased simply because more than the expected number of 50 heads resulted. The reason is that 52 heads 
is consistent with the hypothesis that the coin is fair. On the other hand, flipping 85 or 90 heads in 100 
flips would seem to contradict the hypothesis of a fair coin. In this case there would be a strong case 
for a biased coin. 

In the hybrid-vehicle example, the null hypothesis states that the population parameter of 60 mpg 
has not changed. A second, alternative hypothesis (#,) holds that there has been a change in aver- 
age mpg (i.e., the sample statistic of 64 indicates the population value probably is no longer 60). The 
alternative hypothesis is the logical opposite of the null hypothesis. 

The hybrid-car example can be explored further to show how these concepts are used to test for 
significance: 


° The null hypothesis (H,): There has been no change from the 60 mpg average. 


The alternative hypothesis (H,) may take several forms, depending on the objective of the research- 
ers. The H, may be of the “not the same” or the “greater than” or “less than” form: 


e The average mpg has changed from 60. 


e The average mpg has increased (decreased) from 60. 


These types of alternative hypotheses correspond with two-tailed and one-tailed tests. A two-tailed 
test, or nondirectional test, considers two possibilities: the average could be more than 60 mpg, or it 
could be less than 60. To test this hypothesis, the regions of rejection are divided into two tails of the 
distribution. A one-tailed test, or directional test, places the entire probability of an unlikely outcome 
into the tail specified by the alternative hypothesis. In Exhibit 17-2, the first diagram represents a non- 
directional hypothesis, and the second is a directional hypothesis of the “greater than” variety. 


>Exhibit 17-2 Two- and One-Tailed Tests at the 5 percent Level of Significance 


~<Reject Hyp >!«—— Do not reject Hy ——>'<-Reject Hp > <—— Do not reject Hy) ————> Reject H) > 


025 


Z=-1.96 


60 
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A contender for the 2012 Car of the Year, and a winner in 2010, the 2013 Toyota Prius made news by bringing out 
an array of models, including the V offering the space of an SUV. The Prius is the preeminent hybrid gas-electric car 
and inspires a cult-like devotion which has translated into unprecedented satisfaction rates in user studies. Its most 
fuel-efficient model produces an EPA fuel economy of 51 mpg city and 60 mpg highway. And its 2013 version made 
additional news for delivering the convenience of plug-in charging from any standard household outlet. 
www.prius.com 


Hypotheses for Exhibit 17-2 may be expressed in the following form: 


Null H,: » = 60 mpg 

Alternative H,,: ~ # 60 mpg (not the same case) 
Or 

Null H,: & = 60 mpg 

Alternative H,,: ~ > 60 mpg (greater than case) 
Or 

Null H,: » 2 60 mpg 


Alternative H,: 1 <60 mpg (less than case) 


In testing these hypotheses, adopt this decision rule: take no corrective action if the analysis shows that 
one cannot reject the null hypothesis. Note the language “cannot reject” rather than “accept” the null 
hypothesis. It is argued that a null hypothesis can never be proved and therefore cannot be “accepted.” 
Here, again, we see the influence of inductive reasoning. Unlike deduction, where the connections between 
premises and conclusions provide a legitimate claim of “conclusive proof,” inductive conclusions do not 
possess that advantage. Statistical testing gives only a chance to (1) disprove (reject) or (2) fail to reject the 
hypothesis. Despite this terminology, it is common to hear “accept the null” rather than the clumsy “fail to 
reject the null.” In this discussion, the less formal accept means “fail to reject” the null hypothesis. 

If we reject a null hypothesis (finding a statistically significant difference), then we are accepting the 
alternative hypothesis. In either accepting or rejecting a null hypothesis, we can make incorrect deci- 
sions. A null hypothesis can be accepted when it should have been rejected or rejected when it should 
have been accepted. 

These problems are illustrated with an analogy to the American legal system.' In our system of jus- 
tice, the innocence of an indicted person is presumed until proof of guilt beyond a reasonable doubt can 
be established. In hypothesis testing, this is the null hypothesis; there should be no difference between 
the presumption of innocence and the outcome unless contrary evidence is furnished. Once evidence 
establishes beyond reasonable doubt that innocence can no longer be maintained, a just conviction is 
required. This is equivalent to rejecting the null hypothesis and accepting the alternative hypothesis. 
Incorrect decisions or errors are the other two possible outcomes. We can unjustly convict an innocent 
person, or we can acquit a guilty person. 
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Testing a Hypothesis for Troy-Bilt® 


Marcus Thomas LLC recently conducted an online survey to 
help Troy-Bilt®, a leading manufacturer of lawn and garden 
equipment, understand what elements in television advertising 
help build recall and consideration, specifically within the lawn 
and garden category. 

Marcus Thomas formulated the hypothesis that “consumers 
who are in the market to purchase a product process television 
advertising differently than those who are not in the market for 
lawn and garden equipment,” shared Edwige Winans, associ- 
ate director of research for Marcus Thomas. Carefully selected 
participants were segmented into groups of equal size, based on 
whether or not they were planning to purchase lawn or garden 
equipment (in-market group vs. out-of-market group). To control 
for category-specific reactions, Marcus Thomas also recruited a 
control group, those who were in the market for paint and those 
who were not. 

One thousand respondents were asked to watch an eight- 
minute segment of the DIY Network show “House Crashers” on 
their computer (to simulate television watching). Four 30-second 


television ads were embedded following the first four minutes of 
the show (two for lawn or garden equipment and two for paint), 
followed by four additional minutes of the show. The study was 
blind, in that respondents did not know its purpose. Ad order 
was randomized for each respondent to eliminate order bias. 
In a subsequent online survey, respondents provided both 
aided and unaided recall measurements of the television spots 
(to assess which elements in the ads were most memorable), 
as well as information on how the ads affected perceptions of 
each brand featured and brand awareness. Several red-herring 
questions were introduced to ensure participants would not be 
swayed by the perceived purpose of the study or the merchan- 
dise category. 

The test did showcase unique differences in the way con- 
sumers process television advertising based on whether they are 
in-market or not-in-market for lawn and garden products. 

The survey Marcus Thomas used is available for download 
from the Online Learning Center for your textbook. 


www.marcusthomasllc.com; www.troybilt.com 
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Correct decision 
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° | Probability = 1 - 0 


Type Il error 
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Innocent Guilty 

of crime of crime 
Found Unjustly 
not guilty acquitted 
Innocent Guilty 
Unjustly Justly 
convicted convicted 


Decision: Accept 


Type | error 
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Hg is true 
State of Nature 
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Exhibit 17-3 compares the statistical situation to the legal one. One of two conditions exists in 
nature—either the null hypothesis is true or the alternative hypothesis is true. An indicted person is 
innocent or guilty. Two decisions can be made about these conditions: one may accept the null hypoth- 
esis or reject it (thereby accepting the alternative). Two of these situations result in correct decisions; 
the other two lead to decision errors. 

When a Type I error (c) is committed, a true null hypothesis is rejected; the innocent person is unjustly 
convicted. The value is called the level of significance and is the probability of rejecting the true null. With 
a Type II error (3), one fails to reject a false null hypothesis; the result is an unjust acquittal, with the 
guilty person going free. In our system of justice, it is more important to reduce the probability of convict- 
ing the innocent than that of acquitting the guilty. Similarly, hypothesis testing places a greater emphasis 
on Type I errors than on Type II errors. Next we shall examine each of these errors in more detail. 


Type | Error 


Assume the hybrid manufacturer’s problem is complicated by a consumer testing agency’s assertion that 
the average city mpg has changed. Assume the population mean is 50 mpg, the standard deviation of the 
population is 10 mpg, and the size of the sample is 25 vehicles. With this information, one can calculate the 
standard error of the mean (o- ,) (the standard deviation of the distribution of sample means). This hypo- 
thetical distribution is pictured in Exhibit 17-4. The standard error of the mean is calculated to be 2 mpg: 


>Exhibit 17-4 Probability of Making a Type | Error Given H, Is True 
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Region of 
acceptance 
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If the decision is to reject H, with a 95 percent confidence interval (a = .05), a Type I error of .025 
in each tail is accepted (this assumes a two-tailed test). In part A of Exhibit 17-4, see the regions of 
rejection indicated by the shaded areas. The area between these two regions is known as the region of 
acceptance. The dividing points between the rejection and acceptance areas are called critical values. 
Since the distribution of sample means is normal, the critical values can be computed in terms of the 
standardized random variable,” where 


Z = 1.96 (significance level = .05) 
x = the critical value of the sample mean 
}. = the population value stated in H, = 50 


o; = the standard error of a distribution of means of samples of 25 


Thus, the critical values for the test of the null hypothesis (that the mpg has not changed) are com- 
puted as follows: 


Xt 

Z= 3 

og 
a3 

X = 46.08 

gga 
ng 

X = 53.92 


If the probability of a Type I error is 5 percent (a = .05), the probability of a correct decision if 
the null hypothesis is true is 95 percent. By changing the probability of a Type I error, you move 
critical values either closer to or farther away from the assumed parameter of 50. This can be done 
if a smaller or larger @ error is desired and critical values are moved to reflect this. You can also 
change the Type | error and the regions of acceptance by changing the size of the sample. For ex- 
ample, if you take a sample of 100, the critical values that provide a Type I error of .05 are 48.04 
and 51.96. 

The alternative hypothesis concerned a change in either direction from 50, but the manufacturer 
is interested only in increases in mpg. For this, one uses a one-tailed (greater than) H, and places the 
entire region of rejection in the upper tail of the distribution. One can accept a 5 percent a risk and 
compute a new critical value (X,). (See Appendix D, Exhibit D-1, to find the Z value of 1.645 for the 
area of .05 under the curve.) Substitute this in the Z equation and solve for Xx: 

X. — 50 


Z = 1.645 = 5) 


X= 33.20 


This new critical value, the boundary between the regions of acceptance and rejection, is pictured in 
part B of Exhibit 17-4. 


Type Il Error 


The manufacturer would commit a Type II error (B) by accepting the null hypothesis (u = 50) when in 
truth it had changed. This kind of error is difficult to detect. The probability of committing a B error de- 
pends on five factors: (1) the true value of the parameter, (2) the a level we have selected, (3) whether 
a one- or two-tailed test was used to evaluate the hypothesis, (4) the sample standard deviation, and 
(5) the size of the sample. We secure a different B error if the new B moves from 50 to 54 rather than 
only to 52. We must compute separate B error estimates for each of a number of assumed new population 
parameters and Xx, values. 
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>Exhibit 17-5 Probability of Making a Type II Error 


u = 50 


<———— Don’t reject Hy Reject Hy > 


53.29 


To illustrate, assume wu has actually moved to 54 from 50. Under these conditions, what is the prob- 
ability of our making a Type II error if the critical value is set at 53.29? (See Exhibit 17-5.) This may 
be expressed in the following fashion: 


P(A,)S, = a = .05 (assume a one-tailed alternative hypothesis) 


P(A,)S, =B=? 
_-_ o0 _ 10 _ 
BS AIDS 
xXx- _ 
- gt = 5329 54 _ _ 455 


Using Exhibit D-1 in Appendix D, we interpolate between .35 and .36 Z scores to find the .355 Z score. 
The area between the mean and Z is .1387. B is the tail area, or the area below the Z, and is calculated as 


B = .50 — .1387 = .36 


This condition is shown in Exhibit 17-5. It is the percent of the area where we would not reject the null 
(H,:4 = 50) when, in fact, it was false because the true mean was 54. With an a of .05 and a sample of 
25, there is a 36 percent probability of a Type II (8) error if the p is 54. We also speak of the power of 
the test—that is (1 — 8). For this example, the power of the test equals 64 percent (1 — .36)—that is, 
we will correctly reject the false null hypothesis with a 64 percent probability. A power of 64 percent 
is less than the 80 percent minimum percentage recommended by statisticians. 

There are several ways to reduce a Type II error. We can shift the critical value closer to the original 
u of 50; but to do this, we must accept a bigger a. Whether to take this action depends on the evalua- 
tion of the relative a and £ risks. It might be desirable to enlarge the acceptable a risk because a wors- 
ening of the mileage would probably call for increased efforts to stimulate efficiency. Committing a 
Type I error would mean only that we engaged in efforts to stimulate efficiency when the situation had 
not worsened. This act probably would not have many adverse effects even if mpg had not increased. 

A second way to reduce Type II error is to increase sample size. For example, if the sample were 
increased to 100, the power of the test would be much stronger: 


fe 
x va \/100 


X= po 53.29 = 54 
=" = _ 


B = 50 — .2612 = .24 


1 


Z —71 
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This would reduce the Type II error to 24 percent and increase the power of the test to 76 percent. 

A third method seeks to improve both a and 8 errors simultaneously and is difficult to accomplish. 
We know that measuring instruments, observations, and recording produce error. By using a better 
measuring device, tightening the observation and recording processes, or devising a more efficient 
sample, we can reduce the variability of observations. This diminishes the standard error of estimate 
and in turn reduces the sampling distributions’ spread. The net effect is that there is less tail area in the 
error regions. 


Statistical Testing Procedures 


Testing for statistical significance follows a relatively well-defined pattern, although authors differ in 
the number and sequence of steps. One six-stage sequence is as follows: 


1. State the null hypothesis. Although the researcher is usually interested in testing a hypothesis 
of change or differences, the null hypothesis is always used for statistical testing purposes. 


2. Choose the statistical test. To test a hypothesis, one must choose an appropriate statistical 
test. There are many tests from which to choose, and there are at least four criteria that can be 
used in choosing a test. One is the power efficiency of the test. A more powerful test provides 
the same level of significance with a smaller sample than a less powerful test. In addition, 
in choosing a test, one can consider how the sample is drawn, the nature of the population, 
and (importantly) the type of measurement scale used. For instance, some tests are useful 
only when the sequence of scores is known or when observations are paired; other tests are 
appropriate only if the population has certain characteristics; still other tests are useful only if 
the measurement scale is interval or ratio. More attention is given to test selection later in the 
chapter. 


3. Select the desired level of significance. The choice of the level of significance should be made 
before we collect the data. The most common level is .05, although .01 is also widely used. 
Other levels such as .10, .025, or .0O1 are sometimes chosen. The exact level to choose is 
largely determined by how much risk one is willing to accept and the effect that this choice has 
on B risk. The larger the a, the lower is the B. 


4. Compute the calculated difference value. After the data are collected, use the formula for the 
appropriate significance test to obtain the calculated value. Although the computation typically 
results from a software program, we illustrate the procedures in this chapter to help you visual- 
ize what is being done. 


5. Obtain the critical test value. After we compute the calculated ¢, y*, or other measure, we must 
look up the critical value in the appropriate table for that distribution (or it is provided with the 
software calculation). The critical value is the criterion that defines the region of rejection from 
the region of acceptance of the null hypothesis. 


6. Interpret the test. For most tests if the calculated value is larger than the critical value, we 
reject the null hypothesis and conclude that the alternative hypothesis is supported (although it 
is by no means proved). If the critical value is larger, we conclude we have failed to reject the 
null? 


Probability Values (0 Values) 


According to the “interpret the test” step of the statistical test procedure, the conclusion is stated in 
terms of rejecting or not rejecting the null hypothesis based on a reject region selected before the test is 
conducted. A second method of presenting the results of a statistical test reports the extent to which the 
test statistic disagrees with the null hypothesis. This method has become popular because analysts want 
to know what percentage of the sampling distribution lies beyond the sample statistic on the curve, and 
most statistical computer programs report the results of statistical tests as probability values (p values). 
The p value is the probability of observing a sample value as extreme as, or more extreme than, the 
value actually observed, given that the null hypothesis is true. This area represents the probability of 
a Type I error that must be assumed if the null hypothesis is rejected. The p value is compared to the 
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As this Lieberman Research Worldwide 
ad indicates, a researcher’s responsibility 
goes further than tallying data and 
running the right analytical tests. They 
have to deliver the right conclusions 
from the data. As the ad states, a much- 
demanded researcher will get “the right 
answers to the right decision makers” to 
catapult their success. 
www.Irwonline.com 


so what @ 


It’s a simple question. 
One that demands both answers and action. 
so what? 


It drives our every move, ensuring our research insights 
deliver powerful business-building impact. 


so what? 


By constantly asking this question, we've been guiding 
the right decisions for over 33 years. 


As one of the world’s largest and fastest-growing full-service 
custom research companies, LRW has helped a long list 
of industry-leading businesses improve their bottom-line. 


so what? 


Smart research that delivers great results—it’s exactly what 

makes us so popular. Our team of highly experienced research, 
marketing, and business consultants search for insight in every study. 
But, unlike most research firms, we don’t stop there. We look at the 
results and ask “so what?” — playing out every possible scenario. 
Perhaps it’s right-brain meets left-brain—or maybe it’s our unique 
blend of marketing strategists and analytical leaders. 


Whatever you call it, we get the right answers to the right decision 
makers to catapult your success. 


so what? 


We're not just talking —we’ve got proof. In an arena where 
consistent success is rare, LRW is # 1 worldwide. Independent 
research shows we rate # 1 in top box for overall customer 
satisfaction—again and again. 


Lieberman Research Worldwide Turning Insight into Impact: 


significance level («), and on this basis the null hypothesis is either rejected or not rejected. If the p 
value is less than the significance level, the null hypothesis is rejected (if p value < a, reject the null). 
If p is greater than or equal to the significance level, the null hypothesis is not rejected (if p value > a, 
don’t reject the null). 

Statistical data analysis programs commonly compute the p value during the execution of a hypoth- 
esis test. The following example will help illustrate the correct way to interpret a p value. 

In part B of Exhibit 17-4 the critical value was shown for the situation in which the manufacturer 
was interested in determining whether the average mpg had increased. The critical value of 53.29 was 
computed based on a standard deviation of 10, sample size of 25, and the manufacturer’s willingness to 
accept a 5 percent a risk. Suppose that the sample mean equaled 55. Is there enough evidence to reject 
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the null hypothesis? If the p value is less than .05, the null hypothesis will be rejected. The p value is 
computed as follows. 
The standard deviation of the distribution of sample means is 2. The appropriate Z value is 


X-p 
2 

_ 55 — 50 
4=—"9 
Z=25 


The p value is determined using the standard normal table. The area between the mean and a Z value of 
2.5 is .4938. For this one-tailed test, the p value is the area above the Z value. The probability of observ- 
ing a Z value at least as large as 2.5 is only .0062 (.5000 — .4938 = .0062) if the null hypothesis is true. 

This small p value represents the risk of rejecting a true null hypothesis. It is the probability of a 
Type I error if the null hypothesis is rejected. Since the p value (p = .0062) is smaller than a = .05, 
the null hypothesis is rejected. The manufacturer can conclude that the average mpg has increased. The 
probability that this conclusion is wrong is .0062. 


> Tests of Significance 


This section provides an overview of statistical tests that are representative of the vast array available 
to the researcher. After a review of the general types of tests and their assumptions, the procedures for 
selecting an appropriate test are discussed. The remainder of the section contains examples of para- 
metric and nonparametric tests for one-sample, two-sample, and k-sample cases. Readers needing a 
comprehensive treatment of significance tests are referred to the suggested readings for this chapter. 


Types of Tests 


There are two general classes of significance tests: parametric and nonparametric. Parametric tests are 
more powerful because their data are derived from interval and ratio measurements. Nonparametric 
tests are used to test hypotheses with nominal and ordinal data. Parametric techniques are the tests of 
choice if their assumptions are met. Assumptions for parametric tests include the following: 


e The observations must be independent—that is, the selection of any one case should not affect 
the chances for any other case to be included in the sample. 


e The observations should be drawn from normally distributed populations. 
e These populations should have equal variances. 


e The measurement scales should be at least interval so that arithmetic operations can be used 
with them. 


The researcher is responsible for reviewing the assumptions pertinent to the chosen test. Performing 
diagnostic checks on the data allows the researcher to select the most appropriate technique. The nor- 
mality of a distribution may be checked in several ways. We have previously discussed the measures 
of location, shape, and spread for preliminary analysis and considered graphic techniques for exploring 
data patterns and examining distributions. Another diagnostic tool is the normal probability plot. This 
plot compares the observed values with those expected from a normal distribution. If the data display 
the characteristics of normality, the points will fall within a narrow band along a straight line. An ex- 
ample is shown in the upper left panel of Exhibit 17-6. 

An alternative way to look at this is to plot the deviations from the straight line. These are shown 
in a “detrended” plot in the upper right panel of the figure. Here we would expect the points to clus- 
ter without pattern around a straight line passing horizontally through 0. In the bottom two panels of 
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Exhibit 17-6, there is neither a straight line in the normal probability plot nor a random distribution of 
points about 0 in the detrended plot. Visually, the bottom two plots tell us the variable is not normally 
distributed. In addition, two separate tests of the hypothesis that the data come from normal distribu- 


tions are rejected at a significance level of less than .01.° 


If we wished to check another assumption—say, one of equal variance—a spread-and-level plot 
would be appropriate. Statistical software programs often provide diagnostic tools for checking as- 
sumptions. These may be nested within a specific statistical procedure, such as analysis of variance or 
regression, or provided as a general set of tools for examining assumptions. 
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Testing a Hypothesis of Unrealistic Drug Use in Movies 


Are American teens exposed to unrealistic drug usage or to un- 
realistic consequences from such use? The Office of National 
Drug Control Policy completed a content analysis of the top 
200 rental movies to determine their depiction of substance use. 
The researchers used the Entertainment Merchants Associa- 
tion’s most popular (top 100) home video titles based on rental 
income during two sequential years. Movies were categorized 
as follows: action adventure, comedy, or drama. Data were also 
collected on each title’s Motion Picture Association of America 
(MPAA) rating (G, PG, PG-13, or R). Although technically teens 
should have been excluded from R-rated titles (which made up 
48 percent of the overall sample), the study included all 20 of 
the most popular teen movies as identified in a prior indepen- 
dent study. 

Trained coders watched all 200 movies, paying particular at- 
tention to alcohol, tobacco, illicit drugs, over-the-counter medi- 
cines, prescription medicines, inhalants, and unidentified pills. 


in a hospital or health-related scenario. Substance use included 
explicit portrayals of consumption. Substance appearance was 
noted when evidence of materials or paraphernalia was noted 
without any indication of use. Coders identified dominant mes- 
sages about substance use and the consequences of use. Cod- 
ers also noted scenes depicting illicit drug use or those depicting 
use by characters known to be under 18. Prevalence of use 
was determined by counting the characters in each movie and 
determining not only the percentage of characters using drugs 
but also whether the character had a major or minor role. Cod- 
ers profiled characters by age, gender, and ethnicity, as well as 
other characteristics. Frequency of substance abuse was ceter- 
mined for each five-minute interval of each movie, with the pres- 
ence or absence of various substances noted, starting with the 
completion of the title credits and ending when the final credits 
began. How would the last movie you watched have fared under 
this scrutiny? 


Coders ignored substances administered by medical personnel www.whitehouse.gov/ondcp; www.vsda.org 


Parametric tests place different emphasis on the importance of assumptions. Some tests are quite 
robust and hold up well despite violations. For others, a departure from linearity or equality of variance 
may threaten the validity of the results. 

Nonparametric tests have fewer and less stringent assumptions. They do not specify normally 
distributed populations or equality of variance. Some tests require independence of cases; others are 
expressly designed for situations with related cases. Nonparametric tests are the only ones usable with 
nominal data; they are the only technically correct tests to use with ordinal data, although parametric 
tests are sometimes employed in this case. Nonparametric tests may also be used for interval and ratio 
data, although they waste much of the information available. Nonparametric tests are also easy to 
understand and use. Parametric tests have greater efficiency when their use is appropriate, but even 
in such cases nonparametric tests often achieve an efficiency as high as 95 percent. This means the 
nonparametric test with a sample of 100 will provide the same statistical testing power as a parametric 
test with a sample of 95. 


How to Select a Test 


In attempting to choose a particular significance test, the researcher should consider at least three 
questions: 

° Does the test involve one sample, two samples, or k (more than two) samples? 

e If two samples or k samples are involved, are the individual cases independent or related? 

e Is the measurement scale nominal, ordinal, interval, or ratio? 


Additional questions may arise once answers to these are known: What is the sample size? If there 
are several samples, are they of equal size? Have the data been weighted? Have the data been 
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transformed? Often such questions are unique to the selected technique. The answers can compli- 
cate the selection, but once a tentative choice is made, standard statistics textbooks will provide 
further details. 

Decision trees provide a more systematic means of selecting techniques. One widely used guide 
from the Institute for Social Research starts with questions about the number of variables, nature of the 
variables (continuous, discrete, dichotomous, independent, dependent, and so forth), and level of mea- 
surement. It goes through a tree structure asking detailed questions about the nature of the relationships 
being searched, compared, or tested. More than 130 solutions to data analysis problems are paired with 
commonly asked questions.® 

An expert system offers another approach to choosing appropriate statistics. Capitalizing on the 
power and convenience of personal computers, expert system programs provide a comprehensive 
search of the statistical terrain just as a computer search of secondary sources does. Most programs 
ask about your research objectives, the nature of your data, and the intended audience for your final 
report. When you are not 100 percent confident of your answers, you can bracket them with an estimate 
of the degree of your certainty. One such program, Statistical Navigator™ covers various categories 
of statistics from exploratory data analysis through reliability testing and multivariate data analysis. 
In response to your answers, a report is printed containing recommendations, rationale for selections, 
references, and the statistical packages that offer the suggested procedure.’ SPSS and SAS include 
coaching and help modules with their software. 


Selecting Tests Using the Choice Criteria 


In the next section, we use the three questions discussed in the last section (see bullets) to develop 
a classification of the major parametric and nonparametric tests and measures. Because parametric 
tests are preferred for their power when their assumptions are met, we discuss them first in each of 
the subsections: one-sample tests, two-sample tests, k- (more-than-two) sample tests. This is shown 
in Exhibit 17-7.8 To illustrate the application of the criteria to test selection, consider that your testing 
situation involves two samples, the samples are independent, and the data are interval. The figure sug- 
gests the t-test of differences as the appropriate choice. The most frequently used of the tests listed in 
Exhibit 17-7 are covered next. For additional examples see Appendix C. 


>Exhibit 17-7 Recommended Statistical Techniques by Measurement Level and Testing Situation 


Two-Samples Tests k-Samples Tests 


Measurement Related Independent Related Independent 
Scale One-Sample Case Samples Samples Samples Samples 


Nominal e Binomial e McNemar e Fisher exact test e Cochran Q e \ for k samples 


e y* one-sample test e y° two-samples test 


Interval and e t-test e t-test for paired e t-test e Repeated- e One-way 
Ratio samples measures ANOVA 
e Ztest e Ztest ANOVA 


e n-way ANOVA 
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One-Sample Tests 


One-sample tests are used when we have a single sample and wish to test the hypothesis that it comes 
from a specified population. In this case we encounter questions such as these: 


e Is there a difference between observed frequencies and the frequencies we would expect, based 
on some theory? 


e Is there a difference between observed and expected proportions? 


e Is it reasonable to conclude that a sample is drawn from a population with some specified distri- 
bution (normal, Poisson, and so forth)? 


* Is there a significant difference between some measures of central tendency (X) and its popula- 
tion parameter (1)? 


A number of tests may be appropriate in this situation. The parametric test is discussed first. 


Parametric Tests 


The Z test or ¢-test is used to determine the statistical significance between a sample distribution mean 
and a parameter. 

The Z distribution and ¢ distribution differ. The t has more tail area than that found in the normal 
distribution. This is a compensation for the lack of information about the population standard deviation. 
Although the sample standard deviation is used as a proxy figure, the imprecision makes it necessary 
to go farther away from 0 to include the percentage of values in the ¢ distribution necessarily found in 
the standard normal. 

When sample sizes approach 120, the sample standard deviation becomes a very good estimate of 
the population standard deviation (o); beyond 120, the ¢ and Z distributions are virtually identical. 

Some typical real-world applications of the one-sample test are: 


e Finding the average monthly balance of credit card holders compared to the average monthly 
balance five years ago. 

¢ Comparing the failure rate of computers in a 20-hour test of quality specifications. 

e Discovering the proportion of people who would shop in a new district compared to the assumed 
population proportion. 


e Comparing the average product revenues this year to last year’s revenues. 


Example To illustrate the application of the t-test in the one-sample case, consider again the 
hybrid-vehicle problem mentioned earlier. With a sample of 100 vehicles, the researchers find that 
the mean miles per gallon for the car is 52.5 mpg, with a standard deviation of 14. Do these results 
indicate the population mean might still be 50? 

In this problem, we have only the sample standard deviation (s). This must be used in place of the 
population standard deviation (7). When we substitute s for o, we use the f distribution, especially if 
the sample size is less than 30. We define t as 


This significance test is conducted by following the six-step procedure recommended earlier: 
1. Null hypothesis. 
H,: = 50 miles per gallon (mpg) 
H,: > 50 mpg (one-tailed test) 


2. Statistical test. Choose the t-test because the data are ratio measurements. Assume the under- 
lying population is normal and we have randomly selected the sample from the population of 
production vehicles. 


3. Significance level. Let a = .05, with n = 100. 
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4. Calculated value. 


52.5-—50 _ 2.5 
t= = 1.786 d.f.=n-1=99 
14V100 «14 
5. Critical test value. We obtain this by entering the table of critical values of f (see Appendix C, 
Exhibit C-2, at the back of the book), with 99 degrees of freedom (d.f.) and a level of signifi- 
cance value of .05. We secure a critical value of about 1.66 (interpolated between d.f. = 60 and 
d.f. = 120 in Exhibit D-2). 
6. Interpretation. In this case, the calculated value is greater than the critical value 
(1.786 > 1.66), so we reject the null hypothesis and conclude that the average mpg has 
increased. 


Nonparametric Tests 


In a one-sample situation, a variety of nonparametric tests may be used, depending on the measure- 
ment scale and other conditions. If the measurement scale is nominal (classificatory only), it is possible 
to use either the binomial test or the chi-square (y”) one-sample test. The binomial test is appropriate 
when the population is viewed as only two classes, such as male and female, buyer and nonbuyer, and 
successful and unsuccessful, and all observations fall into one or the other of these categories. The bi- 
nomial test is particularly useful when the size of the sample is so small that the x’ test cannot be used. 


Chi-Square Test 


Probably the most widely used nonparametric test of significance is the chi-square (y7) test. It is 
particularly useful in tests involving nominal data but can be used for higher scales. Typical are cases 
where persons, events, or objects are grouped in two or more nominal categories such as “‘yes—no,” 
“favor—undecided—against,” or class “A, B, C, or D.” 

Using this technique, we test for significant differences between the observed distribution of data 
among categories and the expected distribution based on the null hypothesis. Chi-square is useful in 
cases of one-sample analysis, two independent samples, or k independent samples. It must be calcu- 
lated with actual counts rather than percentages. 

In the one-sample case, we establish a null hypothesis based on the expected frequency of objects 
in each category. Then the deviations of the actual frequencies in each category are compared with the 
hypothesized frequencies. The greater the difference between them, the less is the probability that these 
differences can be attributed to chance. The value of x” is the measure that expresses the extent of this 
difference. The larger the divergence, the larger is the x” value. 

The formula by which the 7” test is calculated is 


k 
2 (O,-E,)° 


in which 
O, = observed number of cases categorized in the ith category 
E, = expected number of cases in the ith category under H, 


k = the number of categories 


There is a different distribution for x? for each number of degrees of freedom (d.f.), defined as 
(k — 1) or the number of categories in the classification minus 1: 


df.=k-1 


With chi-square contingency tables of the two-samples or k-samples variety, we have both rows and 
columns in the cross-classification table. In that instance, d.f. is defined as rows minus | (r — 1) times 
columns minus | (c — 1): 


d.f. =(r— 1c - 1) 
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Ina2 X 2 table there is 1 d.f., and in a3 X 2 table there are 2 d.f. Depending on the number of degrees 
of freedom, we must be certain the numbers in each cell are large enough to make the x” test appropri- 
ate. When d.f. = 1, each expected frequency should be at least 5 in size. If d.f. > 1, then the 7” test 
should not be used if more than 20 percent of the expected frequencies are smaller than 5 or when any 
expected frequency is less than 1. Expected frequencies can often be increased by combining adja- 
cent categories. Four categories of freshmen, sophomores, juniors, and seniors might be classified into 
upper class and lower class. If there are only two categories and still there are too few in a given class, 
it is better to use the binomial test. 

Assume a survey of student interest in the Metro University Dining Club (discussed in Chapter 15) 
is taken. We have interviewed 200 students and learned of their intentions to join the club. We would 
like to analyze the results by living arrangement (type and location of student housing and eating 
arrangements). The 200 responses are classified into the four categories shown in the accompanying 
table. 


(o} E = Expected 
Intend Number Percent (no. Frequencies 
Living Arrangement to Join Interviewed interviewed/200) (percent x 60) 
Dorm/fraternity 16 90 45 27 
Apartment/rooming 
house, nearby 13 40 20 12 
Apartment/rooming 
house, distant 16 40 20 12 
Live at home 16 30 15 
Total 60 200 100 60 


Do these variations indicate there is a significant difference among these students, or are they sampling 
variations only? Proceed as follows: 


1. Null hypothesis. H,: O, = E, The proportion in the population who intend to join the club is 
independent of living arrangement. In H,: O, # E, the proportion in the population who intend 
to join the club is dependent on living arrangement. 


2. Statistical test. Use the one-sample y* to compare the observed distribution to a hypothesized 
distribution. The x” test is used because the responses are classified into nominal categories and 
there are sufficient observations. 


3. Significance level. Leta = .05. 


4. Calculated value. 
k 
(O.-EY 
Hd = L L 


Calculate the expected distribution by determining what proportion of the 200 students interviewed 
were in each group. Then apply these proportions to the number who intend to join the club. Then 
calculate the following: 


— 2 = 2 = 2 —_ 2 
gal? 27) 4 G3 12) , “6 12) 4 ds 9) 


27 12 12 9 
= 4.48 + 0.08 + 1.33 + 4.0 
= 9.89 


df. = (4-12 - 1) =3 
5. Critical test value. Enter the table of critical values of x” (see Exhibit D-3), with 3 d.f., and 
secure a value of 7.82 for a = .05. 


6. Interpretation. The calculated value (9.89) is greater than the critical value (7.82), so the 
null hypothesis is rejected and we conclude that intending to join is dependent on living 
arrangement. 


A/B Testing Comes of Age 
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Split-run testing of direct mail ads has been a feature of design 
for more than 100 years. In direct mail ads, the envelope would 
be changed to see if more people would open the envelope, 
or the offer (2-for-the-cost-of-one versus 50% off) would be 
changed for part of the sample to see if more people would buy. 
Color of the paper, the nature of the salutation, the address on 
the return mail envelope—any or some combination could be the 
focus of a test. But never has split-sample testing been a hotter 
research topic than with the current design of websites. 

Google got the whole thing started in about 2000 with its 
A/B tests. As Optimizely profiles the process, when people visit 
a website, some are diverted to a slightly different version of the 
landing page. For example maybe the “buy” button is blue rather 
than red. Then the visitors to the regular page versus the al- 
tered page are tracked. Does the altered page gain more clicks, 
or longer time spent on the site, or—the holy grail—more pur- 
chases? If so, then the new page will displace the former page. 

There are countless aspects to a page that can be altered, 
one at a time or in tandem with other changes: where text is lo- 
cated, its size and color; what headlines might be used; whether 
still images or video are used; the color scheme of the page; the 
number of links on the page; or where ads are placed on a site. 
The list of possible changes to test is endless. 

As research is always being done on a Web page, we need 
to think about what we are seeing when we visit a page. Are we 


seeing a test page or the standard page that the masses are 
seeing? The Amazon checkout screen | see may be different 
than the one you are seeing. 

So can anyone with a Web page use A/B testing? The an- 
swer just a few years ago would be no, but a former Google 
employee has created Optimizely, a software that allows non- 
programmers to drag, resize, retype, replace, insert, and delete 
at will to create alternate versions of a page to test. 

A/B testing can virtually eliminate a deadly error in decision 
making: having the highest-paid person in a meeting make a 
design decision based on opinion alone. A/B testing makes 
it possible to make all decisions based on data. But because 
small incremental changes are being tested, a danger lurks. 
When bold change is needed, the A/B testing mentality may 
tie the hands of the company webmaster. As Brian Christian, 
author of The Most Human Human: What Artificial Intelligence 
Teaches Us about Being Alive, writing for Wired put it, “Test- 
ing allows you to constantly react to user preferences, but that 
doesn’t necessarily make you agile; 10,000 ongoing tweaks 
don’t add up to a fundamental change of [strategic] direction 
when one is needed.” 


www.optimizely.com; www.wired.com; www.google.com; 
brchristian.com 
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Two-Independent-Samples Tests 


The need to use two-independent-samples tests is often encountered in business research. We might 
compare the purchasing predispositions of a sample of subscribers from two magazines to discover if 
they are from the same population. Similarly, a test of distribution methods from two channels or the 
market share movements from two competing products could be compared. 


Parametric Tests 


The Z and f-tests are frequently used parametric tests for independent samples, although the F test also 
can be used. 

The Z test is used with large sample sizes (exceeding 30 for both independent samples) or with 
smaller samples when the data are normally distributed and population variances are known. The for- 
mula for the Z test is 


ye (X, — X,) — (H, — H,)0 


With small sample sizes, normally distributed populations, and the assumption of equal population 


variances, the f-test is appropriate: oe 
ne (X, — X,) — (ph, — B,)O 
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where 
(,—}.,) is the difference between the two population means. 


S’ is associated with the pooled variance estimate: 
; = se = 1S 
P n,tn,— 2 
To illustrate this application, consider a problem that might face a manager at KDL, a media firm 
that is evaluating account executive trainees. The manager wishes to test the effectiveness of two meth- 
ods for training new account executives. The company selects 22 trainees, who are randomly divided 
into two experimental groups. One receives type A and the other type B training. The trainees are then 
assigned and managed without regard to the training they have received. At the year’s end, the manager 
reviews the performances of employees in these groups and finds the following results: 


A Group B Group 


Average hourly sales _X-bar, = $1,500 _—_-X-bar, = $1,300 
Standard deviation @, = 225 S, = 2s 


Following the standard testing procedure, we will determine whether one training method is supe- 
rior to the other: 


1. Null hypothesis. 
H,;: There is no difference in sales results produced by the two training methods. 


H,: Training method A produces sales results superior to those of method B. 


2. Statistical test. The t-test is chosen because the data are at least interval and the samples are 
independent. 


3. Significance level. a = .05 (one-tailed test). 


4. Calculated value. 
(1,500 — 1,300) — 0 


(es + COED | 1,1 | 
20 ll 11 


= 1.97 


_ 200 
101.63 


There are n — | degrees of freedom in each sample, so total d.f. is 
df. = (11 —1)+ 1-1) = 20 


5. Critical test value. Enter Appendix D, Exhibit D-2 with d.f. = 20, one-tailed test, a = .05. The 
critical value is 1.725. 


6. Interpretation. Since the calculated value is larger than the critical value (1.97 > 1.725), reject 
the null hypothesis and conclude that training method A is superior. 


Nonparametric Tests 


The chi-square (y’) test is appropriate for situations in which a test for differences between samples is 
required. It is especially valuable for nominal data but can be used with ordinal measurements. When 
parametric data have been reduced to categories, they are frequently treated with x’ although this re- 
sults in a loss of information. Preparing to solve this problem is the same as presented earlier although 
the formula differs slightly: 


4 


(0, = £,)° 
2 ij ij 
,, pS E, 


7 os 
J 
in which 


0; = observed number of cases categorized in the ijth cell 


p= expected number of cases under H, to be categorized in the ijth cell 
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Suppose MindWriter is implementing a smoke-free workplace policy and is interested in whether 
smoking affects worker accidents. Since the company has complete reports on on-the-job accidents, a 
sample of names of workers is drawn from those who were involved in accidents during the last year. A 
similar sample from among workers who had no reported accidents in the last year is drawn. Members 
of both groups are interviewed to determine if each is a nonsmoker or smoker and, if a smoker, whether 
the person classifies himself or herself as a heavy or moderate smoker. The results appear in the follow- 
ing table, with expected values calculated as shown. 


On-the-Job Accident 


Cell Designation Count 


Expected Values Yes No Total 
Smoker allel 1,2 

Heavy smoker 127 4 16 
8.24 7.75 
2,1 2,2 

Moderate 9 6 15 
7.73 C27 
3,1 3,2 

Nonsmoker 13 22 35 
18.03 16.97 

Column Total 34 82 66 


The testing procedure is: 
1. Null hypothesis. 


H,: There is no relationship in on-the-job accident occurrences between smokers and nonsmokers. 


H,: There is a relationship in on-the-job accident occurrences between smokers and nonsmokers. 


2. Statistical test. x’ is appropriate, but it may waste some of the data because the measurement 
appears to be ordinal. 


3. Significance level. a = .05, with d.f. = (3 — 1)(2- 1) =2 


4. Calculated value. The expected distribution is provided by the marginal totals of the table. If 
there is no relationship between accidents and smoking, there will be the same proportion of 
smokers in both accident and nonaccident groups. The numbers of expected observations in 
each cell are calculated by multiplying the two marginal totals common to a particular cell and 
dividing this product by n. For example, 


ete = 8.24, the expected value in cell (1,1) 
> U2- 8.24) ae (4 — 7.75) re (9 — 7.73) m (6 — 7.72) 
~ 8.24 7.75 73 ip) 
, (13 = 18.03 | (22 ~ 16.97 
18.03 16.97 


= 7.01 


5. Critical test value. Turn to Appendix D, Exhibit D-3, and find the critical value 7.01 with a = 
.O5 and d.f. = 2. 


6. Interpretation. Since the calculated value is greater than the critical value, the null hypothesis 
is rejected. 


For chi-square to operate properly, data must come from random samples of multinomial distributions, 
and the expected frequencies should not be too small. We previously noted the traditional cautions that 
expected frequencies (E,) below 5 should not compose more than 20 percent of the cells, and that no cell 
should have an E, of less than 1. Some research has argued that these restrictions are too severe.” 
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>Exhibit 17-8 Comparison of Corrected and Noncorrected Chi-Square Results Using SPSS 
Procedure Cross-Tab 


INCOME BY POSSESSION OF MBA 
MBA 
Count 
Yes No 
Row 
1 2 Total 
INCOME 
High 1 30 30 bO 
0.0 
Low 2 10 30 40 
40.0 
Column 40 bO 100 
Total 40.0 bO.0 100.0 
Chi-Square Value D-F- Significance 
Pearson 6.25000 1 01242 
Continuity Correction Sie ] 1 : 
Likelihood Ratio b.4378b 1 
Mantel-Haenszel 6.18750 1 
Minimum Expected Frequency: 16.000 


In another type of x’, the 2 X 2 table, a correction known as Yates’ correction for continuity is 
applied when sample sizes are greater than 40 or when the sample is between 20 and 40 and the val- 
ues of E, are 5 or more. (We use this correction because a continuous distribution is approximating a 
discrete distribution in this table. When the E£,’s are small, the approximation is not necessarily a good 
one.) The formula for this correction is 


n{|AD — BC|- Bl 
X= (A + BY(C + D\(A + CB FD) 


where the letters represent the cells designated as 


A B 
C D 


When the continuity correction is applied to the data shown in Exhibit 17-8, a x” value of 5.25 is ob- 
tained. The observed level of significance for this value is .02192. If the level of significance had been 
set at .01, we would accept the null hypothesis. However, had we calculated x” without correction, the 
value would have been {6/259 which has an observed level of significance of {012423 Some researchers 
may be tempted to reject the null at this level. (But note that the critical value of x at .01 with 1 d.f. is 
6.64. See Appendix D, Exhibit D-3.) The literature is in conflict regarding the merits of Yates’ correc- 
tion, but if nothing else, this example suggests one should take care when interpreting 2 X 2 tables.'° To 
err on the conservative side would be in keeping with our prior discussion of Type I errors. 

The Mantel-Haenszel test and the likelihood ratio also appear in Exhibit 17-8. The former is used 
with ordinal data; the latter, based on maximum likelihood theory, produces results similar to Pearson’s 
chi-square. 


Two-Related-Samples Tests 


The two-related-samples tests concern those situations in which persons, objects, or events are closely 
matched or the phenomena are measured twice. One might compare the consumption of husbands and 
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wives, the performance of employees before and after vacations, or the effects of a marketing test 
stimulus when persons were randomly assigned to groups and given pretests and posttests. Both para- 
metric and nonparametric tests are applicable under these conditions. 


Parametric Tests 


The t-test for independent samples would be inappropriate for this situation because one of its assump- 
tions is that observations are independent. This problem is solved by a formula in which the difference 
is found between each matched pair of observations, thereby reducing the two samples to the equiva- 
lent of a one-sample case—that is, there are now several differences, each independent of the other, for 
which one can compute various statistics. 

In the following formula, the average difference D corresponds to the normal distribution when the 
a difference is known and the sample size is sufficient. The statistic t with (n — 1) degrees of freedom 
is defined as 


+= —_D 
Siva 
where 
p=22 
n 
S —. 


To illustrate, we use two years of Forbes sales data (in millions of dollars) from 10 companies, as 
listed in Exhibit 17-9. 
1. Null hypothesis. 
H,: u = 0; there is no difference between year | and year 2 sales. 
H,: « # O; there is a difference between year | and year 2 sales. 


2. Statistical test. The matched- or paired-samples t-test is chosen because there are repeated mea- 
sures on each company, the data are not independent, and the measurement is ratio. 


>Exhibit 17-9 Sales Data for Paired-Samples t-Test (dollars in millions) 


Sales Sales 

Company Year 2 Year 1 Difference D D? 

GM 126932 123505 3427 11744329 
GE 54574 49662 4912 24127744 
Exxon 86656 78944 WAZ 59474944 
IBM 62710 59512 3198 10227204 
Ford 96146 92300 3846 14791716 
AT&T. 36112 35173 939 881721 
Mobil 50220 48111 2109 4447881 
DuPont 35099 32427 2672 71389584 
Sears 53794 49975 3819 14584761 
Amoco 23966 20779 3187 10156969 


=D = 35821 xD? = 157576853 
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>Exhibit 17-10 SPSS Output for Paired-Samples t-Test (dollars in millions) 


---t-tests for paired samples--- 


Number Standard Standard 
Variable of Cases Mean Deviation Error 
Year 2 Sales 10 bebeo.4 3177? .6b449 10048.975 
Year 1 Sales 10 59034-4 31072.471 9483b.104 
(Difference Standard Standard e-tail t Degrees of e2-tail 
Mean) Deviation Error Corr: Prob- Value Freedom Prob- 
352.1000 1803.159 570.209 | 998 © .o00_| baa q 000 


3. Significance level. Leta = .01, withn = 10 anddf. =n — 1. 


4. Calculated value. 


D_ _ 358210 __ 657 af. 


‘<3 /Va 570.98 


II 
Ne) 


5. Critical test value. Enter Appendix D, Exhibit D-2, with d.f. = 9, two-tailed test, a = .01. The 
critical value is 3.25. 


6. Interpretation. Since the calculated value is greater than the critical value (6.27 > 3.25), reject 
the null hypothesis and conclude there is a statistically significant difference between the two 
years of sales. 


A computer solution to the problem is illustrated in Exhibit 17-10. Notice that an observed sig- 
nificance level is printed for the calculated ¢ value (highlighted). With SPSS, this is often rounded 
and would be interpreted as significant at the .0005 level. The correlation coefficient, to the left of the 
t value, is a measure of the relationship between the two pairs of scores. In situations where matching 
has occurred (such as husbands’ and wives’ scores), it reveals the degree to which the matching has 
been effective in reducing the variability of the mean difference. 


Nonparametric Tests 


The McNemar test may be used with either nominal or ordinal data and is especially useful with before- 
after measurement of the same subjects. Test the significance of any observed change by setting up a 
fourfold table of frequencies to represent the first and second set of responses: 


Before Do Not Favor 
Favor A B 
Do Not Favor Cc D 


Since A + D represents the total number of people who changed (B and C are no-change responses), 
the expectation under a null hypothesis is that 1/2 (A + D) cases change in one direction and the same 
proportion in the other direction. The McNemar test uses a transformation of the x’ test: 


2 = (An DY with df.=1 


The “minus 1” in the equation is a correction for continuity since the y is a continuous distribution and 
the observed frequencies represent a discrete distribution. 
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To illustrate this test’s application, we use survey data from SteelShelf Corporation, whose re- 
searchers decided to test a new concept in office seating with employees at the company’s headquarters 
facility. Managers took a random sample of their employees before the test, asking them to complete a 
questionnaire on their attitudes toward the design concept. On the basis of their responses, the employ- 
ees were divided into equal groups reflecting their favorable or unfavorable views of the design. After 
the campaign, the same 200 employees were asked again to complete the questionnaire. They were 
again classified as to favorable or unfavorable attitudes. The testing process is: 


1. Null hypothesis. 
H,: P(A) = P(D) 
H,: P(A) # P(D) 


2. Statistical test. The McNemar test is chosen because nominal data are used and the study in- 
volves before-after measurements of two related samples. 


3. Significance level. Let a = .05, with n = 200. 
4. Calculated value. 


= = 2 
¥ = (/10 40| 1) = 29? = 16.82 d.f.=1 


10 + 40 50 


Before Do Not Favor 
Favor A= 10 B=90 
Do Not Favor C=60 D=40 


5. Critical test value. Enter Appendix D, Exhibit D-3, and find the critical value to be 3.84 with 
a = .05 and d.f. = 1. 


6. Interpretation. The calculated value is greater than the critical value (16.82 > 3.84), indicat- 
ing one should reject the null hypothesis, and conclude that the new concept had a significant 
positive effect on employees’ attitudes. In fact, x” is so large that it would have surpassed an a 
of .OO1. 


k-Independent-Samples Tests 


We often use k-independent-samples tests in research when three or more samples are involved. 
Under this condition, we are interested in learning whether the samples might have come from the 
same or identical populations. When the data are measured on an interval-ratio scale and we can meet 
the necessary assumptions, analysis of variance and the F test are used. If preliminary analysis shows 
the assumptions cannot be met or if the data were measured on an ordinal or nominal scale, a nonpara- 
metric test should be selected. 

As with the two-samples case, the samples are assumed to be independent. This is the condition of a 
completely randomized experiment when subjects are randomly assigned to various treatment groups. 
It is also common for an ex post facto study to require comparison of more than two independent 
sample means. 


Parametric Tests 


The statistical method for testing the null hypothesis that the means of several populations are equal 
is analysis of variance (ANOVA). One-way analysis of variance is described in this section. It uses 
a single-factor, fixed-effects model to compare the effects of one treatment or factor (brands of cof- 
fee, varieties of residential housing, types of retail stores) on a continuous dependent variable (coffee 
consumption, hours of TV viewing, shopping expenditures). In a fixed-effects model, the levels of 
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the factor are established in advance, and the results are not generalizable to other levels of treatment. 
For example, if coffee were Jamaican-grown, Colombian-grown, and Honduran-grown, we could not 
extend our inferences to coffee grown in Guatemala or Mexico. 

To use ANOVA, certain conditions must be met. The samples must be randomly selected from nor- 
mal populations, and the populations should have equal variances. In addition, the distance from one 
value to its group’s mean should be independent of the distances of other values to that mean (indepen- 
dence of error). ANOVA is reasonably robust, and minor variations from normality and equal variance 
are tolerable. Nevertheless, the analyst should check the assumptions with the diagnostic techniques 
previously described. 

Analysis of variance, as the name implies, breaks down or partitions total variability into component 
parts. Unlike the t-test, which uses sample standard deviations, ANOVA uses squared deviations of the 
variance so that computation of distances of the individual data points from their own mean or from the 
grand mean can be summed (recall that standard deviations sum to zero). 

In an ANOVA model, each group has its own mean and values that deviate from that mean. Simi- 
larly, all the data points from all of the groups produce an overall grand mean. The total deviation is the 
sum of the squared differences between each data point and the overall grand mean. 

The total deviation of any particular data point may be partitioned into between-groups variance 
and within-groups variance. The between-groups variance represents the effect of the treatment, or 
factor. The differences of between-groups means imply that each group was treated differently, and 
the treatment will appear as deviations of the sample means from the grand mean. Even if this were 
not so, there would still be some natural variability among subjects and some variability attributable 
to sampling. The within-groups variance describes the deviations of the data points within each group 
from the sample mean. This results from variability among subjects and from random variation. It is 
often called error. 

Intuitively, we might conclude that when the variability attributable to the treatment exceeds the 
variability arising from error and random fluctuations, the viability of the null hypothesis begins to 
diminish. And this is exactly the way the test statistic for analysis of variance works. 

The test statistic for ANOVA is the F ratio. It compares the variance from the last two sources: 


__ between-groups variance _ mean square,,....., 
within-groups variance mean square in 


where 


sum of squares 


between 


Mean square = 
q between —_ degrees of freedom 


between 


sum of squares 


within 


Mean square... = 
quare, thin degrees of freedom 


within 

To compute the F ratio, the sum of the squared deviations for the numerator and denominator are 
divided by their respective degrees of freedom. By dividing, we are computing the variance as an 
average or mean; thus the term mean square. The degrees of freedom for the numerator, the mean 
square between groups, are one less than the number of groups (k — 1). The degrees of freedom for the 
denominator, the mean square within groups, are the total number of observations minus the number 
of groups (n — k). 

If the null hypothesis is true, there should be no difference between the population means, 
and the ratio should be close to 1. If the population means are not equal, the numerator should 
manifest this difference, and the F ratio should be greater than |. The F distribution determines 
the size of ratio necessary to reject the null hypothesis for a particular sample size and level of 
significance. 

To illustrate one-way ANOVA, consider Travel Industry Magazine’s reports from international 
travelers about the quality of in-flight service on various carriers from the United States to Asia. Before 
writing a feature story coinciding with a peak travel period, the magazine decided to retain a researcher 
to secure a more balanced perspective on the reactions of travelers. The researcher selected passengers 
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>Exhibit 17-11 Data Table: Analysis of Variance Examples 
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Seat Selection? 


Flight Service Flight Service 
Rating 1 Rating 2 Airlinet Seat Selection? Rating 1 Rating 2 Airlinet 
1 40 36 1 1 31 52 65 2 
2 28 28 al 1 32 70 80 2 
3 36 30 1 1 33 73 79 2 
4 382 28 1 if 34 72 838 2 
5 60 40 1 1 35 73 89 z 
6 12 14 1 1 36 al 72 2 
iG 32 26 1 1 37 55 58 Z 
8 36 30 1 1 38 68 67 2 
g) 44 38 1 1 39 81 85 2 
10 36 35 1 1 40 78 80 2 
11 40 42 1 2 4 92 95 3 
2 638 49 1 2 42 56 60 3 
13 20 24 1 2 43 64 70 3 
14 33 35 1 2 44 72 78 3 
15 65 40 1 2 45 48 65 3 
16 40 36 1 2 46 52 70 3 
Ue 51 29 1 2 47 64 79 3 
18 25 24 1 2 48 68 81 3 
19 37 23 1 2 49 76 69 3 
20 44 41 1 2 50 56 78 3 
21 56 67 2 1 51 88 92 3 
22 48 58 2 1 52 i 85 3 
23 64 78 2 1 53 92 94 3 
24 56 68 2 1 54 88 93 3 
25 28 69 2 1 55 73 90 3 
26 32 14 2 1 56 68 67 3 
27 42 55 2 1 57 81 85 3 
28 40 55 2 1 58 95 95 3 
28) 61 80 2 1 59 68 67 3 
30 58 78 2 1 60 78 83 3 


*All data are hypothetical. 
tAirline: 1 = Lufthansa; 2 = Malaysia Airlines; 3 = Cathay Pacific 
#Seat selection: 1 = economy; 2 = business. 


who had current impressions of the meal service, comfort, and friendliness of a major carrier. Three 
airlines were chosen and 20 passengers were randomly selected for each airline. The data, found in 
Exhibit 17-11," are used for this and the next two examples. For the one-way analysis of variance 
problem, we are concerned only with the columns labeled “Flight Service Rating 1” and “Airline.” The 
factor, airline, is the grouping variable for three carriers. 


2 


Rey Rey TRS INSP IRSE IRS) IRS Rey IS 
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>Exhibit 17-12 Summary Tables for One-Way ANOVA Example* 


Model Summaryt 


Source .f. Sum of Squares Mean Square F Value 


Model (airline) 2 11644.033 5822.017 28.304 0.0001 
Residual (error) it 11724.550 205.694 
Total 59 23368.583 


Means Table 


Mean Std. Dev. Std. Error 


Lufthansa 20 38.950 14.006 3.132 
Malaysia Airlines 20 58.900 15.089 3.374 
Cathay Pacific 20 72.900 13.902 3.108 


Scheffé’s S Multiple Comparison Procedure* 


Diff. Crit. Diff. p Value 


Lufthansa Malaysia 19.950 11.400 0.0002 S 
Cathay 33.950 11.400 0.0001 S$ 
Malaysia Cathay 14.000 11.400 0.0122 S 


*All data are hypothetical. 
Factor: airline; dependent: flight service rating 1. 
+S = significantly different at the .05 level; significance level: .05. 


Again, we follow the procedure: 
1. Null hypothesis. 
Ay: May = Hy» =~ Hy; 
H,: Wy, # Hy, # Hy, (The means are not equal.) 


2. Statistical test. The F test is chosen because we have k independent samples, accept the as- 
sumptions of analysis of variance, and have interval data. 


3. Significance level. Let a = .05, and d.f. = [numerator (k — 1) = (3 — 1) = 2], 
[denominator (n — k) = (60 — 3) = 57] = (2,57). 


4. Calculated value. 


MS 
F=55°= ae ae = 28.304 df. (2, 57) 


See summary in Exhibit 17-12. 


5. Critical test value. Enter Appendix D, Exhibit D-8, with d.f. (2, 57), a =.05. The critical value 
is 3.16. 


6. Interpretation. Since the calculated value is greater than the critical value (28.3 > 3.16), we re- 
ject the null hypothesis and conclude there are statistically significant differences between two 
or more pairs of means. Note in Exhibit 17-12 that the p value equals .0001. Since the p value 
(.0001) is less than the significance level (.05), we have a second method for rejecting the null 
hypothesis. 


The ANOVA model summary in Exhibit 17-12 is a standard way of summarizing the results of 
analysis of variance. This table contains the sources of variation, degrees of freedom, sum of squares, 
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mean squares, and calculated F value. The probability of rejecting the null hypothesis is computed up 
to 100 percent a—that is, the probability value column reports the exact significance for the F ratio 
being tested. 


A Priori Contrasts 


When we compute a /-test, it is not difficult to discover the reasons why the null is rejected. But with 

one-way ANOVA, how do we determine which pairs are not equal? We could calculate a series of 

t-tests, but they would not be independent of each other and the resulting Type I error would increase 

substantially. This is not recommended. If we decided in advance that a comparison of specific 

populations was important, a special class of tests known as a priori contrasts could be used after 

the null was rejected with the F test (it is a priori because the decision was made before the test).’” 
A modification of the F' test provides one approach for computing contrasts: 


MS, on 


F-S, 


The denominator, the within-groups mean square, is the same as the error term of the one-way’s F ratio 
(recorded in the summary table, Exhibit 17-12). We have previously referred to the denominator of the 
F ratio as the error variance estimator. The numerator of the contrast test is defined as 


MS oy = SS Fo%) 


CON CON 


c 
a 
where 


C= the contrast coefficient for the group j 


n, = the number of observations recorded for group j 


A contrast is useful for experimental and quasi-experimental designs when the researcher is interested 
in answering specific questions about a subset of the factor. For example, in a comparison of coffee 
products, we have a factor with six levels. The levels, blends of coffee, are meaningfully ordered. As- 
sume we are particularly interested in two Central American—grown blends and one Colombian blend. 
Rather than looking at all possible combinations, we can channel the power more effectively by stating 
the comparisons of interest. This increases our likelihood of detecting differences if they really exist. 


Multiple Comparison Tests 


For the probabilities associated with the contrast test to be properly used in the report of our findings, 
it is important that the contrast strategy be devised ahead of the testing. In the airline study, we had 
no theoretical reason for an a priori contrast. However, when we examine the table of mean ratings 
(Exhibit 17-12), it is apparent that the airline means were quite different. Comparisons after the results 
are compared require post hoc tests or pairwise multiple comparison tests (or range tests) to determine 
which means differ. These tests find homogeneous subsets of means that are not different from each other. 
Multiple comparisons test the difference between each pair of means and indicate significantly different 
group means at an a level of .05, or another level that you specify. Multiple comparison tests use group 
means and incorporate the MS. term of the F ratio. Together they produce confidence intervals for the 
population means and a criterion score. Differences between the mean values may be compared. 

There are more than a dozen such tests with different optimization goals: maximum number of com- 
parisons, unequal cell size compensation, cell homogeneity, reduction of Type I or Type II errors, and so 
forth. The merits of various tests have produced considerable debate among statisticians, leaving the re- 
searcher without much guidance for the selection of a test. In Exhibit 17-13, we provide a general guide. 
For the example in Exhibit 17-12, we chose Scheffé’s S. It is a conservative test that is robust to violations 
of assumptions.'* The computer calculated the critical difference criterion as 11.4; all the differences 
between the pairs of means exceed this. The null hypothesis for the Scheffé was tested at the .05 level. 
Therefore, we can conclude that all combinations of flight service mean scores differ from each other. 
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>Exhibit 17-13 Selection of Multiple Comparison Procedures 


Equal Unequal 
Pairwise Complex Variances Variances Not 

Comparisons Comparisons Equal n’s Only Unequal n’s Assumed Assumed 
Fisher LSD Xx X Xx 
Bonferroni Xx X Xx 
Tukey HSD Xx X Xx 
Tukey-Kramer X X Xx 
Games-Howell Xx X X 
Tamhane T2 X Xx X 
Scheffé S xX xX x 
Brown-Forsythe Xx Xx Xx 
Newman-Keuls Xx Xx Xx 
Duncan Xx Xx Xx 
Dunnett’s T3 X 
Dunnett’s C Xx 


While the table in Exhibit 17-12 provides information for understanding the rejection of the one- 
way null hypothesis and the Scheffé null, in Exhibit 17-14 we use plots for the comparisons. The means 
plot shows relative differences among the three levels of the factor. The means by standard deviations 
plot reveals lower variability in the opinions recorded by the hypothetical Lufthansa and Cathay Pacific 
passengers. Nevertheless, these two groups are sharply divided on the quality of in-flight service, and 
that is apparent in the plot. 


Exploring the Findings with Two-Way ANOVA 


Is the airline on which the passengers traveled the only factor influencing perceptions of in-flight 
service? By extending the one-way ANOVA, we can learn more about the service ratings. There are 
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Means Plot Scattergram of Cell Means versus Standard Deviation 
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>Exhibit 17-15 Summary Table for Two-Way ANOVA Example* 


Model Summaryt 


Source -f. Sum of Squares Mean Square F Value p Value 


Airline 2 11644.033 5822.017 39.178 0.0001 
Seat selection 1 3182.817 3182.817 21.418 0.0001 
Airline by seat selection 2 517.033 258.517 1.740 0.1853 
Residual 54 8024.700 148.606 


Means Table Effect: Airline by Seat Selection 


Count Mean Std. Dev. Std. Error 
Lufthansa economy 10 35.600 12.140 3.839 
Lufthansa business 10 42.300 15.550 4.917 
Malaysia economy 10 48.500 12.501 3.953 
Malaysia business 10 69.300 9.166 2.898 
Cathay economy 10 64.800 13.037 4.123 
Cathay business 10 81.000 9.603 3.0387 
*All data are hypothetical. 


‘Dependent: Flight service rating 1. 


many possible explanations. We have chosen to look at the seat selection of the travelers in the interest 
of brevity. 

Recall that in Exhibit 17-11, data were entered for the variable seat selection: economy and 
business-class travelers. Adding this factor to the model, we have a two-way analysis of variance. 
Now three questions may be considered with one model: 


e Are differences in flight service ratings attributable to airlines? 
¢ Are differences in flight service ratings attributable to seat selection? 


¢ Do the airline and the seat selection interact with respect to flight service ratings? 


The third question reveals a distinct advantage of the two-way model. A separate one-way model on 
airlines averages out the effects of seat selection. Similarly, a single-factor test of seat selection aver- 
ages out the effects of the airline choice. But an interaction test of airline by seat selection considers 
them jointly. 

Exhibit 17-15 reports a test of the hypotheses for these three questions. The significance level was 
established at the .01 level. We first inspect the interaction effect, airline by seat selection, since the 
individual main effects cannot be considered separately if the factors interact. The interaction was not 
significant at the .01 level, and the null is accepted. Now the separate main effects, airline and seat 
selection, can be verified. As with the one-way ANOVA, the null hypothesis for the airline factor was 
rejected, and seat selection was also rejected (statistically significant at .0001). 

Means and standard deviations listed in the table are plotted in Exhibit 17-16. We note a band of 
similar deviations for economy-class travelers and a band of lower variability for business class—with 
the exception of one carrier. The plot of cell means confirms visually what we already know from the 
summary table: there is no interaction between airline and seat selection (p = .185). If an interaction 
had occurred, the lines connecting the cell means would have crossed rather than displaying a parallel 
pattern. 

Analysis of variance is an extremely versatile and powerful method that may be adapted to a wide 
range of testing applications. Discussions of further extensions in m-way and experimental designs may 
be found in the list of suggested readings in the text’s Online Learning Center. 
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>Exhibit 17-16 Two-Way Analysis of Variance Plots 


Standard deviation of rating 1 
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Nonparametric Tests 


When there are k independent samples for which nominal data have been collected, the chi-square test 
is appropriate. It can also be used to classify data at higher measurement levels, but metric informa- 
tion is lost when reduced. The k-samples y’ test is an extension of the two-independent-samples cases 
treated earlier. It is calculated and interpreted in the same way. 

The Kruskal-Wallis test is appropriate for data that are collected on an ordinal scale or for inter- 
val data that do not meet F-test assumptions, that cannot be transformed, or that for another reason 
prove to be unsuitable for a parametric test. Kruskal-Wallis is a one-way analysis of variance by 
ranks. It assumes random selection and independence of samples and an underlying continuous 
distribution. 

Data are prepared by converting ratings or scores to ranks for each observation being evaluated. The 
ranks range from the highest to the lowest of all data points in the aggregated samples. The ranks are 
then tested to decide if they are samples from the same population. An application of this technique is 
provided in Appendix C. 


k-Related-Samples Tests 


Parametric Tests 


A k-related-samples test is required for situations where (1) the grouping factor has more than two 
levels, (2) observations or subjects are matched or the same subject is measured more than once, and 
(3) the data are at least interval. In test marketing experiments or ex post facto designs with k samples, 
itis often necessary to measure subjects several times. These repeated measurements are called trials. 
For example, multiple measurements are taken in studies of stock prices, products evaluated by reli- 
ability, inventory, sales, and measures of product performance. Hypotheses for these situations may 
be tested with a univariate or multivariate general linear model. The latter is beyond the scope of this 
discussion. 

The repeated-measures ANOVA is a special type of n-way analysis of variance. In this design, 
the repeated measures of each subject are related just as they are in the related t-test when only two 
measures are present. In this sense, each subject serves as its own control requiring a within-subjects 
variance effect to be assessed differently than the between-groups variance in a factor like airline or 
seat selection. The effects of the correlated measures are removed before calculation of the F ratio. 
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This model is an appropriate solution for the data presented in Exhibit 17-11. You will remember 
that the one-way and two-way examples considered only the first rating of in-flight service. Assume a 
second rating was obtained after one week by reinterviewing the same respondents. We now have two 
trials for the dependent variable, and we are interested in the same general question as with the one-way 
ANOVA, with the addition of how the passage of time affects perceptions of in-flight service. 

Following the testing procedure, we state: 

1. Null hypotheses. 

(1) Airline: Ay Bay = ba = Baas 

(2) Ratings: H): Up, = Ups 
(3) Ratings x airline: Ay: (Mirsat Hroa2 Moras) = (Mevat Mara2 Hevas) 
For the alternative hypotheses, we will generalize to the statement that not all the groups have 
equal means for each of the three hypotheses. 


2. Statistical test. The F test for repeated measures is chosen because we have related trials on 
the dependent variable for k samples, accept the assumptions of analysis of variance, and have 
interval data. 


3. Significance level. Let a = .05 and d.f. = [airline (2, 57), ratings (1, 57), ratings by airline 
(2, 57)]. 


4. Calculated values. See summary in Exhibit 17-17. 


>Exhibit 17-17 Summary Tables for Repeated-Measures ANOVA* 


Model Summaryt 


Source .f. Sum of Squares Mean Square F Value p Value 


Airline 2 35527.550 17763.775 67.199 0.0001 
Subject (group) 57 15067.650 264.345 
Ratings 1 625.633 625.633 14.318 0.0004 
Ratings by air....... D 2061.717 1030.858 23.592 0.0001 
Ratings by subj... . . 57 2490.650 43.696 


Means Table Ratings by Airline 


Count Mean Std. Dev. Std. Error 


Rating 1, Lufthansa 20 38.950 14.006 3.132 
Rating 1, Malaysia 20 58.900 15.089 3.374 
Rating 1, Cathay 20 72.900 13.902 3.108 
Rating 2, Lufthansa 20 82.400 8.268 1.849 
Rating 2, Malaysia 20 72.250 10.572 2.364 
Rating 2, Cathay 20 79.800 iteZ265 2,019 


Means Table Effect: Ratings 


Mean Std. Dev. Std. Error 


Rating 1 60 56.917 19.902 2.569 
Rating 2 60 61.483 23.208 2.996 


*All data are hypothetical. 
tDependent: flight service ratings 1 and 2. 
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>Exhibit 17-18 Repeated-Measures ANOVA Plot 


Interaction Plot Effect: Ratings by Airline* 
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*All data are hypothetical. 


5. Critical test value. Enter Appendix D, Exhibit D-8, with d-f. (2, 57), a = .05 and (1, 57), 
a = .05. The critical values are 3.16 (2, 57) and 4.01 (1, 57). 


6. Interpretation. The statistical results are grounds for rejecting all three null hypotheses and 
concluding there are statistically significant differences between means in all three instances. 
We conclude the perceptions of in-flight service were significantly affected by the different 
airlines, the interval between the two measures had a significant effect on the ratings, and the 
measures’ time interval and the airlines interacted to a significant degree. 


The ANOVA summary table in Exhibit 17-17 records the results of the tests. A means table pro- 
vides the means and standard deviations for all combinations of ratings by airline. A second table 
of means reports the differences between flight service ratings | and 2. In Exhibit 17-18, there is an 
interaction plot for these data. Note that the second in-flight service rating was improved in two of the 
three groups after one week, but for the third carrier, there was a decrease in favorable response. The 
intersecting lines in the interaction plot reflect this finding. 


Nonparametric Tests 


When the k related samples have been measured on a nominal scale, the Cochran Q test is a good 
choice.'* This test extends the McNemar test, discussed earlier, for studies having more than two 
samples. It tests the hypothesis that the proportion of cases in a category is equal for several related 
categories. 

When the data are at least ordinal, the Friedman two-way analysis of variance is appropriate. It tests 
matched samples, ranking each case and calculating the mean rank for each variable across all cases. 
It uses these ranks to compute a test statistic. The product is a two-way table where the rows represent 
subjects and the columns represent the treatment conditions.’ See Appendix C for additional nonpara- 
metric tests. 
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ee eS 


1 Inclassical statistics we make inferences about a popula- 
tion based on evidence gathered from a sample. Although 
we cannot state unequivocally what is true about the entire 
population, representative samples allow us to make state- 
ments about what is probably true and how much error is 
likely to be encountered in arriving at a decision. The Bayes- 
ian approach also employs sampling statistics but has an 
additional element of prior information to improve the deci- 
sion maker’s judgment. 


2 Adifference between two or more sets of data is statisti- 
cally significant if it actually occurs in a population. To 
have a statistically significant finding based on sampling 
evidence, we must be able to calculate the probability that 
some observed difference is large enough that there is little 
chance it could result from random sampling. Probability is 
the foundation for deciding on the acceptability of the null 
hypothesis, and sampling statistics facilitate acquiring the 
estimates. 


3 Hypothesis testing can be viewed as a six-step procedure: 


a Establish a null hypothesis as well as the alternative hy- 
pothesis. It is a one-tailed test of significance if the alter- 
native hypothesis states the direction of difference. If no 
direction of difference is given, it is a two-tailed test. 


b Choose the statistical test on the basis of the assumption 
about the population distribution and measurement level. 
The form of the data can also be a factor. In light of these 
considerations, one typically chooses the test that has 
the greatest power efficiency or ability to reduce decision 
errors. 


c Select the desired level of confidence. While a = .05 
is the most frequently used level, many others are also 
used. The a is the significance level that we desire and 
is typically set in advance of the study. Alpha or Type | 
error is the risk of rejecting a true null hypothesis and 
represents a decision error. The 8 or Type Il error is the 
decision error that results from accepting a false null hy- 
pothesis. Usually, one determines a level of acceptable a 
error and then seeks to reduce the £ error by increasing 


the sample size, shifting from a two-tailed to a one-tailed 
significance test, or both. 


d Compute the actual test value of the data. 


e Obtain the critical test value, usually by referring to a 
table for the appropriate type of distribution. 


f Interpret the result by comparing the actual test value 
with the critical test value. 


Parametric and nonparametric tests are applicable under the 
various conditions described in the chapter. They were also 
summarized in Exhibit 17-6. Parametric tests operate with in- 
terval and ratio data and are preferred when their assumptions 
can be met. Diagnostic tools examine the data for violations 
of those assumptions. Nonparametric tests do not require 
stringent assumptions about population distributions and are 
useful with less powerful nominal and ordinal measures. 


In selecting a significance test, one needs to know, at a mini- 
mum, the number of samples, their independence or related- 
ness, and the measurement level of the data. Statistical tests 
emphasized in the chapter were the Z and t-tests, analysis 

of variance, and chi-square. The Z and t-tests may be used 
to test for the difference between two means. The t-test is 
chosen when the sample size is small. Variations on the t-test 
are used for both independent and related samples. 

One-way analysis of variance compares the means of 
several groups. It has a single grouping variable, called a 
factor, and a continuous dependent variable. Analysis of 
variance (ANOVA) partitions the total variation among scores 
into between-groups (treatment) and within-groups (error) 
variance. The F ratio, the test statistic, determines if the 
differences are large enough to reject the null hypothesis. 
ANOVA may be extended to two-way, n-way, repeated- 
measures, and multivariate applications. 

Chi-square is a nonparametric statistic that is used fre- 
quently for cross-tabulation or contingency tables. Its appli- 
cations include testing for differences between proportions 
in populations and testing for independence. Corrections for 
chi-square were discussed. 
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Terms in Review 
1 Distinguish between the following: 


a Parametric tests and nonparametric tests. 
b Type | error and Type II error. 

c Null hypothesis and alternative hypothesis. 
d Acceptance region and rejection region. 

e One-tailed tests and two-tailed tests. 

f Type Il error and the power of the test. 


2 Summarize the steps of hypothesis testing. What is the 
virtue of this procedure? 


3 In analysis of variance, what is the purpose of the mean 
square between and the mean square within? If the null 
hypothesis is accepted, what do these quantities look like? 


4 Describe the assumptions for ANOVA, and explain how 
they may be diagnosed. 


Making Research Decisions 
5 Suggest situations where the researcher should be more 
concerned with Type Il error than with Type | error. 


a How can the probability of a Type | error be reduced? 


A Type II error? 8 


b How does practical significance differ from statistical 
significance? 

c Suppose you interview all the members of the fresh- 
man and senior classes and find that 65 percent of the 
freshmen and 62 percent of the seniors favor a pro- 
posal to send Help Centers offshore. Is this difference 


significant? 9 


6 What hypothesis testing procedure would you use in the 
following situations? 


a Atest classifies applicants as accepted or rejected. On 
the basis of data on 200 applicants, we test the hypoth- 
esis that ad placement success is not related to gender. 


b Acompany manufactures and markets automobiles 
in two different countries. We want to know if the gas 
mileage is the same for vehicles from both facilities. 
There are samples of 45 units from each facility. 


c Acompany has three categories of marketing analysts: 
(1) with professional qualifications but without work 
experience, (2) with professional qualifications and with 
work experience, and (3) without professional qualifica- 
tions but with work experience. A study exists that mea- 
sures each analyst’s motivation level (classified as high, 
normal, and low). A hypothesis of no relation between 
analyst category and motivation is to be tested. 


d Acompany has 24 salespersons. The test must evalu- 
ate whether their sales performance is unchanged or 
has improved after a training program. 


e Acompany has to evaluate whether it should attribute 
increased sales to product quality, advertising, or an 
interaction of product quality and advertising. 


You conduct a survey of a sample of 25 members of this 
year’s graduating marketing students and find that the 
average GPA is 3.2. The standard deviation of the sample 
is 0.4. Over the last 10 years, the average GPA has been 
3.0. Is the GPA of this year’s students significantly different 
from the long-run average? At what alpha level would it be 
significant? 

You are curious about whether the professors and stu- 
dents at your school are of different political persuasions, 
so you take a sample of 20 professors and 20 students 
drawn randomly from each population. You find that 10 
professors say they are conservative and 6 students 

say they are conservative. Is this a statistically significant 
difference? 


You contact a random sample of 36 graduates of Western 
University and learn that their starting salaries averaged 
$28,000 last year. You then contact a random sample of 
40 graduates from Eastern University and find that their 
average starting salary was $28,800. In each case, the 
standard deviation of the sample was $1,000. 


a Test the null hypothesis that there is no difference 
between average salaries received by the graduates of 
the two schools. 


b What assumptions are necessary for this test? 
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10 Arandom sample of students is interviewed to determine if Test the hypothesis that there are no differences between 
there is an association between class and attitude toward the means of these products (a = .05). 
corporations. With the following results, test the hypothesis 13 Acomputer manufacturer is introducing a new product 
that there is no difference among students on this attitude. specifically targeted at the home market and wishes to 
Favorable Neutral Unfavorable compare the effectiveness of three sales strategies: com- 
Feehan 400 50 70 puter stores, home electronics stores, and department 
stores. Numbers of sales by 15 salespeople are recorded 
Sophomores 80 60 70 freee 
ehalers me a0 i Electronics store: 5, 4, 3, 3, 3 
Seniors 40 60 90 


Department store: 9, 7, 8, 6, 5 
11. You do a survey of marketing students and liberal arts Computer store: 7, 4, 8, 4, 3 
school students to find out how many times a week they 
read a daily newspaper. In each case, you interview 100 
students. You find the following: 


a Test the hypothesis that there is no difference between 
the means of the retailers (a = .05). 


b Select a multiple comparison test, if necessary, to 


y= OTIS pales determine which groups differ in mean sales (a = .05). 


S.:= 1:5 
X,, = 5.6 times per week From the Headlines 
S,, = 2.0 14 Employers, military, and colleges use aptitude tests to 


predict how well someone might perform. Recently, crit- 
ics have said there isn’t much difference in performance 
above a certain level—that everyone is more or less the 
same. Now, in a current issue of Psychological Science, 
the authors of a new study find that this isn’t true. Instead, 
the higher your score, the better you perform later. The 
investigation considered four large studies of people who 
have taken aptitude tests: the College Board’s SAT scores 
for 150,000 students entering 110 colleges and their fresh- 
man GPA. The Army collected 5,000 scores for the Armed 
Services Vocational Aptitude Battery and later appraised 


Test the hypothesis that there is no significant difference 
between these two samples. 


12 One-Koat Paint Company has developed a new type of 
porch paint that it hopes will be the most durable on the 
market. The R&D group tests the new product against the 
two leading competing products by using a machine that 
scrubs until it wears through the coating. One-Koat runs 
five trials with each product and secures the following re- 
sults (in thousands of scrubs): 


Trial One-Koat Competitor A Competitor B 7 
candidates on how well they did their jobs. Two additional 
1 37 34 24 ‘ ; : 
data sets contained students’ performance on tests in 
2 30 19 25 high school and their grades in college. The higher the test 
3 34 22 23 scores, the better the subsequent performance. Suggest 
4 28 31 20 alternative hypotheses that could equally explain this 
Oar 
Inquiring Minds Want to Know—NOW! NCRCC: Teeing Up a New Strategic Direction 
Marcus Thomas LLC Tests Hypothesis for Troy- Proofpoint: Capitalizing on a Reporters Love of Statistics 


Bilt Creative Development 
Yahoo!: Consumer Direct Marries Purchase Metrics to 


Banner Ads 
Mastering Teacher Leadership 


* You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine whether a 
case provides data, the research instrument, video, or other supplementary material. Written cases are downloadable from the 
text website (www.mhhe.com/cooper1 2e). All video material and video cases are available from the Online Learning Center. 
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learningobjectives 
After reading this chapter, you should understand .. . 


How correlation analysis may be applied to study relationships between two or more variables. 
The uses, requirements, and interpretation of the product moment correlation coefficient. 


How predictions are made with regression analysis using the method of least squares to minimize errors in drawing 
a line of best fit. 


How to test regression models for linearity and whether the equation is effective in fitting the data. 


The nonparametric measures of association and the alternatives they offer when key assumptions and requirements 
for parametric techniques cannot be met. 


6 6 Consumer behavior with digital editions of magazines 
is very much like their behavior with print editions of 
magazines, and very much unlike their behavior with 
websites. Readers typically swipe through tablet editions 
from front to back, for example, the same way they work 


their way through print editions. They browse—taking in 
ads as they go—instead of jumping directly to specific 


articles the way web surfers do. 9 9 
Scott McDonald, senior vice-president for research and insights, 
Conde Nast 


bringingresearcnhtolife 


Sara arrives for an analysis meeting with Jason and 
finds a round, bald little man sitting at Jason’s desk, 
studying the screen of a laptop computer, stroking his 
gray beard and smiling broadly. 

“Sara,” says Jason, “meet Jack Adams, rising political 
consultant.” 

Jack, who seems to be caressing his laptop, grins 
broadly. “Hello, Sara,” says Jack. “I wanted Jason to 
know this little computer has made me the marketing 
kingpin of the Boca Beach political scene.” 

“Jack sold his painting business on Long Island to 
his three boys and moved to Boca Beach after his wife 
passed away,” offers Jason in explanation. 

“For three months I played golf in the morning and 
sat by the pool and played cards in the afternoon. For 
three months, seven days, I did this. I was going crazy. 
Then my next-door neighbor Marty died and his wife 
gave me his MindWriter.” 

“Jason came through Boca Beach and stopped for a 
visit. He downloaded a statistical program, free from 
the Internet. I must say, statistics in college never 
generated as much excitement as they have recently,” 
grins Jack. “We had a wise guy, Sandy Plover, a 
former electrical contractor in Jersey, who got himself 
into local politics. Being a natural-born troublemaker, 
he waited for his chance to agitate. As it happens, the 
sheriff released data to the newspaper that the incidence 
of arrests resulting from police calls to Oceanside— 
the richer of the two neighborhoods where the sheriff 
happens to live—is higher than in Gladeside.” 

Jack types the following: 

Research hypothesis: Oceanside residents get special 
treatment when it comes to solving crimes and thus live 
in a safer environment due to their higher incomes and 
greater political power. 

Null hypothesis: Gladeside and Oceanside receive the 
same attention from the police. 


Gladeside Oceanside 
Police calls without arrest 46 40 
Police calls resulting in arrest 4 10 
Total calls 50 50 


“T doubt that Sandy would have paid attention, 
except that in both neighborhoods the total number 
of police calls happened to be 50, which made it easy 
for him to see that in Oceanside the rate of arrests was 
twice that in Gladeside.” 

“Actually,” says Sara, “I’m surprised there would 
be any police calls in such an upscale community.” 

“We are old,” says Jack, “but not dead.” 

“Tn any case, Sandy’s finely honed political instincts 
told him he was going nowhere by trying to turn the 
community against the sheriff. It would be much, 
much better to turn voters of Oceanside against those 
in Gladeside. So he complained about the disparate 
impact of arrests. While both the communities are 
roughly the same size, in Oceanside folks are mostly 
from Brooklyn, and in Gladeside folks come from the 
Bronx.” 

“But the ethics .. .” 

“". . meant nothing to Sandy. He told me, ‘I think 
I’m gonna kick some butt and make a name for myself 
down here.’ ” 

“The trouble with the police calls as an issue is 
that sheriffs’ offices nowadays are well staffed with 
statistically educated analysts who know very well how 
to rebut oddball claims,” interrupts Jason. 

“Although I personally miss the old days, I punched 
the numbers into this MindWriter here to double-check 
the stats. I did the obvious first, just what I supposed 
a police analyst would have done. I ran a cross- 
tabulation and a chi-square test of the hypothesis that 
the arrests in Oceanside were disproportionate to those 
in Gladeside.” 


“To an untrained observer it would appear that they 
are,” contributes Sara as she peers over Jack’s shoulder. 
“But then I’m not so easily hoodwinked.” 

“Good for you, Sara!” exclaims Jack. “What you 
have here is the ‘eyeball’ fallacy, as my dear old 
professor called it almost 50 years ago. As I explained 
to Sandy, 100 police encounters resulting in a few 
arrests is nothing, nada, not a large enough sample to 
trust a quick peek and a leap to a conclusion. You run 
it through the computer, and, sure enough, although the 
ratios seem to be out of whack, the difference is not 
statistically significant. You cannot support ‘disparate 
impact.’ No way.” 

“Granting that 10 arrests per 50 is bigger than 4 per 
50,” observes Jason, “Jack saw that a statistician would 
say that it is not disproportionate enough to convince 
a scientist that the police were acting differently in the 
two neighborhoods. A statistician would say, Wait and 
see, let the story unfold, collect a bigger sample.” 

“How did Sandy accept your explanation, Jack?” 

“He was ready to shoot the messenger, very much 
bothered, at first, that I would not support his political 
strategy. But I was pretty sure the sheriff would come 
roaring back with a statistical analysis to throw cold 
water over Sandy.” 

“Did you bring him around?” 

“That jerk, come around? Never. He ran to the papers, 
and spilled his numbers and accusations in a letter to the 
editor that was printed on a Monday, and Tuesday the 
sheriff came back with his experts and made Sandy look 
like a fool—on page one, if you can believe it. So Sandy 
was washed up, but he mentioned to the reporter that I 
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had provided the same interpretation before he went to 
the paper, so I now have a new career: resident political 
genius. What I do is look at the opponents’ polling 
results and deny their validity for the newspapers and 
TV. If the opposing party is ahead by a few poll points, 
I scoff at the thinness of the margin. If their lead is wide, 
I belittle the size of the sample and intimate that any 
statistician would see through them.” 

“Jack is colorful, amusing, and good-natured in 
debunking his opponents’ polls, and the newspaper 
writers have never challenged him to substantiate his 
claims or interpretations,” contributes Jason. “What 
he learned from me is that statistics is so complicated, 
and scares so many people, that you can claim or deny 
anything. And he is usually right to debunk the polls, 
since, fora preelection political poll to be taken seriously, 
there has to be a large enough sample to produce 
significant results. And there has to be a big enough 
spread between winners and losers to protect against a 
last-minute shift in voter sentiment. In the small, closely 
contested voting precincts of condominium politics, 
hardly any poll can meet two such stringent criteria.” 

“So now I sit in my condo’s clubhouse and people 
come over and want to know what I think about the 
Middle East, campaign reform, everything.” Jack, 
rising, extends his hand to Sara, “I can tell you have 
things to do, so I'll move along. It was nice meeting 
you, Sara.” 

Sara watches Jack Adams give Jason a bear hug and 
then walk out the door. 

“So, Sara, what did you think of Jack’s knowledge 
of statistics?” 


In the previous chapter, we emphasized testing hypotheses of difference. However, management ques- 
tions frequently revolve around the study of relationships between two or more variables. Then, a 
relational hypothesis is necessary. In the research question “Are U.S. kitchen appliances perceived by 
American consumers to be of better quality than foreign kitchen appliances?” the nature of the rela- 
tionship between the two variables (“country of origin” and “perceived quality’’) is not specified. The 
implication, nonetheless, is that one variable is responsible for the other. A correct relational hypoth- 
esis for this question would state that the variables occur together in some specified manner without 
implying that one causes the other. 

Various objectives are served with correlation analysis. The strength, direction, shape, and other 
features of the relationship may be discovered. Or tactical and strategic questions may be answered by 
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predicting the values of one variable from those of another. Let’s look at some typical management 
questions: 


e In the mail-order business, excessive catalog costs quickly squeeze margins. Many mailings fail 
to reach receptive or active buyers. What is the relationship between mailings that delete inac- 
tive customers and the improvement in profit margins? 


e Medium-size companies often have difficulty attracting the cream of the MBA crop, and when 
they are successful, they have trouble retaining them. What is the relationship between the can- 
didate’s rank based on an executive interview and the rank obtained from testing or managerial 
assessment? 


e Cigarette company marketing allocations shifted a few years ago as a result of multistate settle- 
ments eliminating outdoor and transit advertising. More recently, advertising in magazines with 
large youth readerships came under scrutiny. During a given period, what is the relationship be- 
tween point-of-sale expenditures and net profits? 


e Aggressive U.S. high-tech companies have advertised heavily in the European chip market, and 
their sales have grown 20 percent over sales of the three largest European firms. Can we predict 
next year’s sales based on present advertising? 


All these questions may be evaluated by means of measures of association. And all call for different 
techniques based on the level at which the variables were measured or the intent of the question. The 
first three use nominal, ordinal, and interval data, respectively. The last one is answered through simple 
linear regression. 

With correlation, one calculates an index to measure the nature of the relationship between vari- 
ables. With regression, an equation is developed to predict the values of a dependent variable. Both are 
affected by the assumptions of measurement level and the distributions that underlie the data. 

Exhibit 18-1 lists some common measures and their uses. The chapter follows the progression of the 
exhibit, first covering bivariate linear correlation, then examining simple regression, and concluding 
with nonparametric measures of association. Exploration of data through visual inspection and diag- 
nostic evaluation of assumptions continues to be emphasized. 


> Bivariate Correlation Analysis 


Bivariate correlation analysis differs from nonparametric measures of association and regression 
analysis in two important ways. First, parametric correlation requires two continuous variables mea- 
sured on an interval or ratio scale. Second, the coefficient does not distinguish between independent 
and dependent variables. It treats the variables symmetrically since the coefficient r has the same 
interpretation as r, . 


Pearson’s Product Moment Coefficient r 


The Pearson (product moment) correlation coefficient varies over a range of +1 through 0 to —1. The 
designation r symbolizes the coefficient’s estimate of linear association based on sampling data. The 
coefficient p represents the population correlation. 

Correlation coefficients reveal the magnitude and direction of relationships. The magnitude is the 
degree to which variables move in unison or opposition. The size of a correlation of +.40 is the same 
as one of —.40. The sign says nothing about size. The degree of correlation is modest. The coeffi- 
cient’s sign signifies the direction of the relationship. Direction tells us whether large values on one 
variable are associated with large values on the other (and small values with small values). When the 
values correspond in this way, the two variables have a positive relationship: as one increases, the 
other also increases. Family income, for example, is positively related to household food expendi- 
tures. As income increases, food expenditures increase. Other variables are inversely related. Large 
values on the first variable are associated with small values on the second (and vice versa). The prices 
of products and services are inversely related to their scarcity. In general, as products decrease in 


469 


470 


>part IV _ Analysis and Presentation of Data 


>Exhibit 18-1 Commonly Used Measures of Association 


Measurement Coefficient 


Comments and Uses 


Pearson (product moment) 
correlation coefficient 
Correlation ratio (eta) 
Biserial 

Partial correlation 


Multiple correlation 


Bivariate linear regression 


For continuous linearly related variables. 
For nonlinear data or relating a main effect to a 
continuous dependent variable. 


One continuous and one dichotomous variable 
with an underlying normal distribution. 


Three variables; relating two with the third’s 
effect taken out. 


Three variables; relating one variable with 
two others. 


Predicting one variable from another’s scores. 


Gamma 


Kendall’s tau b 
Kendall’s tau c 
Somers’s d 


Spearman’s rho 


Based on concordant-discordant pairs: (P — Q); 
proportional reduction in error (PRE) interpretation. 


P—Q based; adjustment for tied ranks. 
P—Q based; adjustment for table dimensions. 
P—Q based; asymmetrical extension of gamma. 


Product moment correlation for ranked data. 


Phi 


Cramer’s V 


Contingency coefficient C 


Lambda 
Goodman & Kruskal’s tau 
Uncertainty coefficient 


Kappa 


available quantity, their prices rise. The absence of a relationship is expressed by a coefficient of 


approximately zero. 


Chi-square (CS) based for 2 x 2 tables. 


CS based; adjustment when one table 
dimension > 2. 


CS based; flexible data and distribution assumptions. 


PRE-based interpretation. 
PRE-based with table marginals emphasis. 
Useful for multidimensional tables. 


Agreement measure. 


Scatterplots for Exploring Relationships 


Scatterplots are essential for understanding the relationships between variables. They provide a means 
for visual inspection of data that a list of values for two variables cannot. Both the direction and the 
shape of a relationship are conveyed in a plot. With a little practice, the magnitude of the relationship 
can be seen. 

Exhibit 18-2 contains a series of scatterplots that depict some relationships across the range r. The 
three plots on the left side of the figure have their points sloping from the upper left to the lower right 
of each x-y plot.! They represent different magnitudes of negative relationships. On the right side of the 
figure, the three plots have opposite patterns and show positive relationships. 
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>Exhibit 18-2 Scatterplots of Correlations between Two Variables 


y y y 


r= -1.00 r=0 r=+4+1.00 


- 1.00 — .50 0 +.50 + 1.00 
Perfect negative No relationship Perfect positive 
relationship relationship 


When stronger relationships are apparent (for example, the +.90 correlations), the points cluster 
close to an imaginary straight line passing through the data. The weaker relationships (+.40) depict a 
more diffuse data cloud with points spread farther from the line. 

The shape of linear relationships is characterized by a straight line, whereas nonlinear relationships 
have curvilinear, parabolic, and compound curves representing their shapes. Pearson’s r measures 
relationships in variables that are linearly related. It cannot distinguish linear from nonlinear data. 
Summary statistics alone do not reveal the appropriateness of the data for the model, which is why 
inspecting the data is important. 

The need for data visualization is illustrated with four small data sets possessing identical summary 
statistics but displaying strikingly different patterns.” Exhibit 18-3 contains these data, and Exhibit 18-4 
plots them. In Plot | of the figure, the variables are positively related. Their points follow a superimposed 
straight line through the data. This example is well suited to correlation analysis. In Plot 2, the data are 
curvilinear in relation to the line, and ris an inappropriate measure of their relationship. Plot 3 shows the 
presence of an influential point that changed a coefficient that would have otherwise been a perfect + 1.0. 
The last plot displays constant values of x (similar to what you might find in an animal or quality control 
experiment). One leverage point establishes the fitted line for these data. 

We will return to these concepts and the process of drawing the line when we discuss regression. 
For now, comparing Plots 2 through 4 with Plot | suggests the importance of visually inspecting cor- 
relation data for underlying patterns to ensure linearity. Careful analysts make scatterplots an integral 
part of the inspection and exploration of their data. Although small samples may be plotted by hand, 
statistical software packages save time and offer a variety of plotting procedures. 
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>Exhibit 18-3 Four Data Sets with the Same Summary Statistics 


Oro SO) oS a 


= 
(S) 


11 


Xx, Y, x, Y, x, Y, 
10 8.04 10 9.14 10 7.46 
8 6.95 8 8.14 8 Cxae 
13 7.58 13 8.74 13 12.74 
¢) 8.81 @) 8.77 ©) Halla 
did 8.33 14 Q25 aul 7.81 
14 S196 14 8.10 14 8.84 
6 7.24 6 6.13 6 6.08 
4 4.26 4 3.10 4 5.39 
12 10.84 12 Ons 12 8.15 
G 4.82 i 7.26 if 6.42 
5 5.68 5 4.74 5 5.73 


>Exhibit 18-4 Different Scatterplots for the Same Summary Statistics 


Plot 1 


Plot 2 
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The Assumptions of r 


Like other parametric techniques, correlation analysis makes certain assumptions about the data. Many 
of these assumptions are necessary to test hypotheses about the coefficient. 

The first requirement for r is linearity. All of the examples in Exhibit 18-2 with the exception of 
r = Oillustrate a relationship between variables that can be described by a straight line passing through 
the data cloud. When r = 0, no pattern is evident that could be described with a single line. Parentheti- 
cally, it is also possible to find coefficients of 0 where the variables are highly related but in a nonlinear 
form. As we have seen, plots make such findings evident. 

The second assumption for correlation is a bivariate normal distribution—that is, the data are 
from a random sample of a population where the two variables are normally distributed in a joint 
manner. 

Often these assumptions or the required measurement level cannot be met. Then the analyst should 
select a nonlinear or nonparametric measure of association, many of which are described later in this 
chapter. 


Computation and Testing of r 


The population correlation coefficient is 


p= =a. (1) 


Because population parameters are usually not known to us, we estimate from the random sample of 
(X, Y) observation pairs. 
With the sample estimator of Cov(X, Y) being SS,, /(n — 1), an estimator of o, is 


\/SS,/n — 1) 


and an estimator of Oo, is 


/SS Jn — 1 


We can substitute these estimators for their population counterparts in equation (2), giving us the 
sample correlation coefficient designated by r. 


SS,, 
(2) 


”" 4/8S,SS, 


Another common formula for calculating Pearson’s r is 


_ >(X¥ -Xyy-Y) 
r GSS, (3) 


where 
n = the number of pairs of cases 


S. S, = the standard deviations for X and Y 


A variation known as the reflective correlation, used when the data are not centered on their mean 
values, is: 


Vergy) 


VF °F 


since 
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>Exhibit 18-5 Computation of Pearson’s Product Moment Correlation 


(2) (3) (4) (5) 


Net Profits Cash Flow 
($, millions) ($, millions) Deviations from Means 
Corporation x Y (X - X)x (Y-Y)y 
1 82.6 126.5 —93.84 —178.64 16763.58 8805.95 31912.25 
2 89.0 191.2 —87.44 —113.94 9962.91 7645.75 12982.32 
3 176.0 267.0 —0.44 —38.14 16.78 0.19 1454.66 
4 82.3 137.1 —94.14 —168.04 15819.29 8862.34 28237.44 
5 413.5 806.8 237.06 501.66 118923.52 56197.44 251602.56 
6 1841 35.2 158.34 —269.94 42742.3 25071.56 72867.60 
7 337.3 425.5 160.86 120.36 19361.114 25875.94 14486.53 
8 145.8 380.0 —30.64 74.86 OOS ial 938.81 5604.02 
8) 172.6 326.6 —3.84 21.36 82.02 14.75 456.25 
10 247.2 355.5 70.76 50.36 3563.47 5006.98 2536.13 
= Ne e130 5a14 yy = DAA ZS 


s— 216.50) 5 124.01 
Dx? = 138419.71 
dy’ = 422139.76 


If the numerator of equation (4) is divided by n, we have the covariance, the amount of deviation that 
the X and Y distributions have in common. With a positive covariance, the variables move in unison; 
with a negative one, they move in opposition. When the covariance is 0, there is no relationship. The 
denominator for equation (4) represents the maximum potential variation that the two distributions 
share. Thus, correlation may be thought of as a ratio. 

Exhibit 18-5 contains a random subsample of 10 firms of the Forbes 500 sample. The variables cho- 
sen to illustrate the computation of r are cash flow and net profits. Beneath each variable is its mean and 
standard deviation. In columns 4 and 5 we obtain the deviations of the X and Y values from their means, 
and in column 6 we find the product. Columns 7 and 8 are the squared deviation scores. 

Substituting into the formula, we get 


22477723 
V/138419.71 * \/422139.76 


9298 


In this subsample, net profits and cash flow are positively related and have a very high coefficient. As 
net profits increase, cash flow increases; the opposite is also true. Linearity of the variables may be 
examined with a scatterplot such as the one shown in Exhibit 18-6. The data points fall along a straight 
line. 


Common Variance as an Explanation 


The amount of common variance in X (net profits) and Y (cash flow) may be summarized by the 
coefficient of determination (r?). As Exhibit 18-7 shows, the overlap between the two variables is the 
proportion of their common or shared variance. 

The area of overlap represents the percentage of the total relationship accounted for by one variable 
or the other. So 86 percent of the variance in X is explained by Y, and vice versa. 
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>Exhibit 18-6 Plot of Forbes 500 Net Profits with Cash Flow 
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>Exhibit 18-7 Diagram of Common Variance 


Testing the Significance of r 


Is the coefficient representing the relationship between net profits and cash flow real, or does it occur 
by chance? This question tries to discover whether our r is a chance deviation from a population p of 
zero. In other situations, the researcher may wish to know if significant differences exist between two 
or more r’s. In either case, r’s significance should be checked before r is used in other calculations or 
comparisons. For this test, we must have independent random samples from a bivariate normal distri- 
bution. Then the Z or t-test may be used for the null hypothesis, p = 0. 


476 


>part IV _ Analysis and Presentation of Data 


The formula for small samples is 


F 
t = 
L=r 
n= 2 
where 
r= 93 
n= 10 
Substituting into the equation, we calculate f: 
(ja = 708 


With n — 2 degrees of freedom, the statistical program calculates the value of t (7.03) at a probability 
less than .005 for the one-tailed alternative, H,: p > 0. We reject the hypothesis that there is no linear 
relationship between net profits and cash flow in the population. The preceding statistic is appropriate 
when the null hypothesis states a correlation of 0. It should be used only for a one-tailed test.* How- 
ever, it is often difficult to know in advance whether the variables are positively or negatively related, 
particularly when a computer removes our contact with the raw data. Software programs produce two- 
tailed tests for this eventuality. The observed significance level for a one-tailed test is half of the printed 


two-tailed version in most programs. 


In the fine art of research, 
the shades of gray complete the masterpiece. 


While data give answers in black and white, it’s the subtleties of the gray areas that give you the big picture. 
Burke understands the nuances of research. Grounded in academic principles and guided by ongoing internal 
research, Burke helps you determine the best research method, gather the information, and develop the best 


strategy for actionable results. You will have confidence in your decisions because you have 
the experts at Burke to support you. Visit Burke.com or call 800.688.2674 to find out more. Burke 


The Fine Art of Marketing Research 


This ad captures data analysis accurately: “While data give answers in 


black and white, it’s the subtleties of the gray areas that give you the big 
picture.” Although a dominant answer to a variable might give you insight 
to the majority of responses, studying the less prevalent responses will 


likely give you a richer understanding of your relevant population. 
www.burke.com 


Interpretation of Correlations 


A correlation coefficient of any magnitude or sign, 
whatever its statistical significance, does not imply 
causation. Increased net profits may cause an increase 
in market value, or improved satisfaction may cause 
improved performance in certain situations, but cor- 
relation provides no evidence of cause and effect. 
Several alternate explanations may be provided for 
correlation results: 


e X causes Y. 
e Ycauses X. 


e X and Y are activated by one or more other 
variables. 


e X and Y influence each other reciprocally. 


Ex post facto studies seldom possess sufficiently pow- 
erful designs to demonstrate which of these conditions 
could be true. By controlling variables under an exper- 
imental design, we may obtain more rigorous evidence 
of causality. 

Take care to avoid so-called artifact correla- 
tions, in which distinct groups combine to give the 
impression of one. The upper panel of Exhibit 18-8 
shows data from two business sectors. If all the data 
points for the X and Y variables are aggregated and 
a correlation is computed for a single group, a posi- 
tive correlation results. Separate calculations for each 
sector (note that points for sector A form a circle, as 
do points for sector B) reveal no relationship between 
the X and Y variables. A second example, shown in 
the lower panel, contains a plot of data on assets and 
sales. We have enclosed and highlighted the data for 
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>Exhibit 18-8 Artifact Correlations 


y 


Sector A# 
Sector Be 


22,500 | 


15,000 


~«— Financial sector 


Assets 


7,500 


0 2,000 4,000 6,000 


the financial sector. This is shown as a narrow band enclosed by an ellipse. The companies in this 
sector score high on assets and low in sales—all are banks. When the data for banks are removed and 
treated separately, the correlation is nearly perfect (.99). When banks are returned to the sample and 
the correlation is recalculated, the overall relationship drops to the middle .80s. In short, data hidden 
or nested within an aggregated set may present a radically different picture. 

Another issue affecting interpretation of coefficients concerns practical significance. Even when a 
coefficient is statistically significant, it must be practically meaningful. In many relationships, other 
factors combine to make the coefficient’s meaning misleading. For example, in nature we expect rain- 
fall and the height of reservoirs to be positively correlated. But in states where water management 
and flood control mechanisms are complex, an apparently simple relationship may not hold. Tech- 
niques like partial and multiple correlation or multiple regression are helpful in sorting out confound- 
ing effects. 

With large samples, even exceedingly low coefficients can be statistically significant. This “‘sig- 
nificance” only reflects the likelihood of a linear relationship in the population. Should magnitudes 
less than .30 be reported when they are significant? It depends. We might consider the correlations 
between variables such as cash flow, sales, market value, or net profits to be interesting revelations of 
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Is Oscar Showing Some Tarnish? 


Each year the movie industry puts its best pictures and its brightest actors, writers, and directors into a competition known as the 
Oscars. Voted on by members of the Academy of Motion Picture Arts and Sciences, the Oscars is not only designed to recognize 
excellence, but also, in a blatant way, to increase viewing of movies. 

Polaris Marketing Research, Inc. and AMC Theatres set out to determine whether the Oscar event had any measurable effect on movie 
viewership. Launching a brief online survey via Research Now’s OmniPulse® omnibus offering, and matching attitudinal and behavior 
questions to extensive demographic data, Polaris Marketing Research, Inc. indicates that for a movie, studio, or actor all the pre-event 
hype and the Oscar event itself influence a relatively small percentage to view an Oscar-nominated or Oscar-winning movie. Do the data 
below indicate that women and men respond differently to Oscar, at a level that is statistically significant? How would you determine this? 


The Question: How much do you agree or disagree with the following statements? 


Strongly Somewhat Somewhat _— Strongly Don’t 


Agree Agree Disagree Disagree Know 
| enjoy following the Oscar nomination process and pre-Oscar buzz. 4 3 2 1 5 
| make it a point to see the movies that get Oscar nominations. 4 3 2 1 5 
| make it a point to see the movies that win an Oscar. 4 3 2 1 5 
The Data: 
Males Females 
Strongly Strongly Strongly Strongly 
Agree/  Disagree/ Agree/ Disagree/ 
Agree Disagree Agree Disagree Total 
| enjoy following the Oscar nomination Count 198 367 316 322 565 males 
process and pre-Oscar buzz. 638 females 
Percent 35.0 65.0 49.5 50.5 
Mean 1.93 males 
2.24 females 
| make it a point to see the movies that Count 199 366 266 372 
get Oscar nominations. 
Percent 35.2 64.8 41.7 58.3 
Mean 1.98 males 
2.13 females 
| make it a point to see the movies that Count 234 331 313 325 
win an Oscar. 
Percent 41.4 58.6 49.1 50.9 
Mean 2.10 males 
2.31 females 


www.polarismr.com, www.amctheatres.com, www.researchnow.com 


a particular phenomenon whether they were high, moderate, or low. The nature of the study, the char- 
acteristics of the sample, or other reasons will be determining factors. A coefficient is not remarkable 
simply because it is statistically significant. 

By probing the evidence of direction, magnitude, statistical significance, and common variance 
together with the study’s objectives and limitations, we reduce the chances of reporting trivial findings. 
Simultaneously, the communication of practical implications to the reader will be improved. 
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>Exhibit 18-9 Comparison of Bivariate Linear Correlation and Regression 


Measurement 
level 


Nature of 
variables 


X-Y 
relationship 


Correlation 


Correlation 


Interval or ratio scale 


Both continuous, 
linearly related 


X and Y are symmetric; 
yy = yx 


The correlation of x 
and y produces an 
estimate of linear 
association based on 


Interval or ratio scale 


Both continuous, 
linearly related 


Y is dependent, X is 
independent; regression of X 
on Y differs from Y on X 


Correlation of Y — X is 

the same as the correlation 
between the predicted values 
of Y and observed values of Y 


sampling data 


Proportion of variability of Y 
explained by its least-squares 
regression on X 


Coefficient of 
determination 


Explains common 
variance of X and Y 


> Simple Linear Regression‘ 


In the previous section, we focused on relationships between variables. The product moment correlation 
was found to represent an index of the magnitude of the relationship, the sign governed the direction, 
and 7° explained the common variance. Relationships also serve as a basis for estimation and prediction. 

When we take the observed values of X to estimate or predict corresponding Y values, the process is 
called simple prediction.° When more than one X variable is used, the outcome is a function of mul- 
tiple predictors. Simple and multiple predictions are made with a technique called regression analysis. 

The similarities and differences of correlation and regression are summarized in Exhibit 18-9. Their 
relatedness would suggest that beneath many correlation problems is a regression analysis that could 
provide further insight about the relationship of Y with X. 


The Basic Model 


A straight line is fundamentally the best way to model the relationship between two continuous vari- 
ables. The bivariate linear regression may be expressed as 


Y= Bo + BX; 


where the value of the dependent variable Y is a linear function of the corresponding value of the in- 
dependent variable X, in the ith observation. The slope and the Y intercept are known as regression 
coefficients. The slope,  ,, is the change in Y for a 1-unit change in X. It is sometimes called the “rise 
over run.” This is defined by the formula 

ne 
This is the ratio of change (A) in the rise of the line relative to the run or travel along the X axis. 
Exhibit 18-10 shows a few of the many possible slopes you may encounter. 
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>Exhibit 18-10 Examples of Different Slopes 


6 


4 Line Slope 
>The test for a 0 a -1 
slope is described b 0 
later, in “Testing the 
Goodness of Fit.” c +1 
2 d ee 


The intercept, {,, is the value for the linear function when it crosses the Y axis; it is the estimate of 
Y when X = 0. A formula for the intercept based on the mean scores of the X and Y variables is 


Bo =Y¥— BX 


Concept Application 


What makes Generation X-ers all over the world select a glass of wine rather than a beer, Jack Daniels 
and Coke, or Bacardi Breezer? A research report from Australia highlights Generation X attitudes 
toward wine. The results suggest the top influencers are friends and family, wine reviews, and visits to 
wineries.° From the winery’s perspective, tasting from the barrel is not only a widespread sales tool but 
also a major determinant of market en primeur or futures contracts, which represent about 60 percent 
of the harvest. 

Weather is widely regarded as responsible for pronouncements about a wine’s taste and potential 
quality. A Princeton economist has elaborated on that notion. He suggested that just a few facts about 
local weather conditions may be better predictors of vintage French red wines than the most refined 
palates and noses.’ The regression model developed predicts an auction price index for about 80 wines 
from winter and harvest rainfall amounts and average growing-season temperatures. Interestingly, the 
calculations suggested that the 1989 Bordeaux would be one of the best since 1893. French traditional- 
ists reacted hysterically to these methods yet agreed with the conclusion. 

Our first example will use one predictor with highly simplified data. Let X represent the average 
growing-season temperature in degrees Celsius and Y the price of a 12-bottle case in euros. Take, 
for example, a famous French burgundy such as Romande Conti St. Vivant, which sells for $340 per 
bottle (times 12 bottles per case or $4,080). This would be approximately 3,060 euros. The data 
appear here: 


Average matte. (Celsius) Price per ae (EUR) 
12 2,000 
16 3,000 
20 4,000 
24 5,000 


x= 18 Wo 500) 
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>Exhibit 18-11 Plot of Wine Price by Average Growing Temperature 
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The plotted data in Exhibit 18-11 show a linear relationship between the pairs of points and a perfect 


positive correlation, r= 1.0. The slope of the line is calculated: 
Y;— ¥, _ 4,000 — 3,000 _ 1,000 
at Fi as 9 2 ere: — 
Bi-x¥ =x ~~ 20= 16 ass 


where the X,Y, values are the data points (20, 4,000) and XY, are points (16, 3,000). The intercept B, 
is — 1,000, the point at which X = 0 in this plot. This area is off the graph and appears in an insert on 
the figure. 


By = Y — B.X = 3,500 — 250(18) = —1,000 
Substituting into the formula, we have the simple regression equation 
Y = —1,000 + 250X; 


We could now predict that a warm growing season with 25.5°C temperature would bring a case price 
of 5,375 euros. Y (called Y-hat) is the predicted value of Y: 


Y¥ = —1,000 + 250(25.5) = 5,375 


Unfortunately, one rarely comes across a data set composed of four paired values, a perfect correla- 
tion, and an easily drawn line. A model based on such data is deterministic in that for any value of X, 
there is only one possible corresponding value of Y. It is more likely that we will collect data where the 
values of Y vary for each X value. Considering Exhibit 18-12, we should expect a distribution of price 
values for the temperature X = 16, another for X = 20, and another for each value of X. The means of 
these Y distributions will also vary in some systematic way with X. These variabilities lead us to con- 
struct a probabilistic model that also uses a linear function.’ This function is written 


Y; = Bo + BiX;, + & 


where e symbolizes the deviation of the ith observation from the mean, B, + B,X,. 
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>Exhibit 18-12 Distribution of Y for Observations of X 


Y 
5,500 


3,000 
2,500 
2,000 


1,500 
10 12 14 16 18 20 22 24 26 


Y=Bo+ BX, +e Probability distribution Y= Bo + BX, +e 


- + 


Regression line Bo + B4X; 


As shown in Exhibit 18-12, the actual values of Y may be found above or below the regression line 
represented by the mean value of Y (8, + 8,X,) for a particular value of X. These deviations are the 
error in fitting the line and are often called the error term. 


Method of Least Squares 


Exhibit 18-13 contains a new data set for the wine price example. Our prediction of Y from X must 
now account for the fact that the X and Y pairs do not fall neatly along the line. Actually, the relation- 
ship could be summarized by several lines. Exhibit 18-14 suggests two alternatives based on visual 
inspection—both of which produce errors, or vertical distances from the observed values to the line. 
The method of least squares allows us to find a regression line, or line of best fit, that will keep these 
errors to a minimum. It uses the criterion of minimizing the total squared errors of estimate. When we 
predict values of Y for each X,, the difference between the actual Y, and the predicted Y is the error. This 
error is squared and then summed. The line of best fit is the one that minimizes the total squared errors 
of prediction.’ 


n 

2: Be lse28 
> e; minimized 
i=1 


Regression coefficients 8, and B, are used to find the least-squares solution. They are computed as 
follows: 


; SXY — Sem 
iS. iq 2 
pc oenearts 


6, -7- Bx 
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>Exhibit 18-13 Data for Wine Price Study 


Price (EUR) Temperature (C) 
Y 

1 1,813 11.80 21,393.40 3,286,969.00 139.24 
2 2,558 15.70 40,160.60 6,543,364.00 246.49 
S 2,628 14.00 36,792.00 6,906,384.00 196.00 
4 SANG 22.90 73,669.30 10,349,089.00 524.41 
6) 3,228 20.00 64,560.00 10,419,984.00 400.00 
6 3,629 20.10 72,942.90 13,169,641 .00 404.01 
i 3,886 17.90 69,559.40 15,100,996.00 320.41 
8 4,897 23.40 114,589.80 23,980,609.00 547.56 
9 4,933 24.60 121,351.80 24,334,489.00 605.16 
10 5,199 25.70 133,614.30 27,029,601 .00 660.49 
s 35,988 196.10 748,633.50 141,121,126.00 4,043.77 
Mean 3,598.80 19.61 

Ss 1,135.66 4.69 

Sum of squares (SS) WA ASOM GSN dS) 198.25 42,908.82 


>Exhibit 18-14 Scatterplot and Possible Regression Lines Based on Visual Inspection: Wine 
Price Study 
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Substituting data from Exhibit 18-13 into both formulas, we get 


(10. DG5.988) = 216.439 


~ (196.1)? 
4,043.77 — 


Bo = 3,598.8 — (216.439)(19.61) = —645.569 
The predictive equation is now Y = —645.57 + 216.44 X;. 


748,633.5 — 


Bi 


Drawing the Regression Line 


Before drawing the regression line, we select two values of X to compute. Using values 13 and 24 for 
X,, the points are 


Y = —645.57 + 216.44(13) = 2,168.15 
Y = —645.57 + 216.44(24) = 4,548.99 


Comparing the line drawn in Exhibit 18-15 to the trial lines in Exhibit 18-14, one can readily see the 
success of the least-squares method in minimizing the error of prediction. 


>picorofile 


Constellation Wines used both qualitative and quantitative research to design its “Count On It” campaign for its Blackstone 
brand. The recent recession caused wine buyers to change their behavior, so rather than purchase based on the adventure 
of discovery, today’s buyers rely on the reduced risk associated with established brands. To take advantage of that change in 
behavior, Constellation Wines launched a multistage research project. First, positioning research used six focus groups—three 
each in Chicago and Los Angeles, half with women and half with men—to reveal how Blackstone wine compared to and con- 
trasted with its nearest competitors. These guided discussions included word sorts. “We chose words we thought might be 
relevant, as well as those we thought were definitely not relevant,” shared the Constellation Wines’ marketing director. One 
unexpected word, masculine, kept surfacing among both men and women. Probing by the moderator revealed this was not 
masculine as macho or threatening, but rather masculine as strength. The focus groups revealed that several other words and 
phases were associated with Blackstone Wine: “unpretentious quality,” “unassuming hero,” “engaging,” as well as “staple 
that’s going to deliver.” Amazon Advertising developed three different ad approaches and tested each to be sure it was true to 
the updated brand positioning. “Given the research findings, we wanted to be sure that our new campaign didn’t come across 
quite as feminine as our last campaign.” Gallup and Robinson then tested multiple ad executions using 20-minute Web-based 
interactive interviews, showing the ad against comparative run-of-press ads in targeted print media and capturing prospect 
recall, reaction, and understanding. The ad featured here is one of two ads that research helped Constellation Wines choose to 
promote Blackstone Wine. www.cbrands.com; www.gallup-robinson.com; www.amazonadv.com 
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>Exhibit 18-15 Drawing the Least-Squares Line: Wine Price Study 
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Residuals 


We now turn our attention to the plot of standardized residuals in Exhibit 18-16. A residual is what 
remains after the line is fit or (Y, — Y,.). When standardized, residuals are comparable to Z scores with 
a mean of 0 and a standard deviation of 1. In this plot, the standardized residuals should fall between 2 
and —2, be randomly distributed about zero, and show no discernible pattern. All these conditions say 


the model is applied appropriately. 


In our example, we have one residual at —2.2, a random distribution about zero, and few indications 
of a sequential pattern. It is important to apply other diagnostics to verify that the regression assump- 
tions (normality, linearity, equality of variance, and independence of error) are met. Various software 


22 24 


programs provide plots and other checks of regression assumptions." 


>Exhibit 18-16 Plot of Standardized Residuals: Wine Price Study 
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1,813 
2,558 
2,628 
3,217 
3,228 
3,629 
3,886 
4,897 
4,933 
5,199 


COON OAHKWNDY 
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Predicted 
Price 


1,908.4112 
2,752.5234 
2,384.5771 
4,310.8844 
3,683.2112 
3,704.8551 
3,228.6893 
4,419.1039 
4,678.8307 
4,916.9137 


26 28 


Residual 


-95.4112 
-194.5234 
243.4229 
—1,093.8844 
—455.2112 
-75.8551 
657.3107 
477.8961 
254.1693 
282.0863 
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Envirosell: Studies Reveal Left-Hand Retail 


World retailers collect and subscribe to numerous 
data sources, but they need knowledge from the 
data to craft their merchandising, staffing, and pro- 
motion strategies, as well as their store designs. 
Retail giants (e.g., The Gap, Limited, Starbucks, 
Radio Shack, McDonald’s) turn to consultant Paco 
Underhill when they want to know how consum- 
ers buy what they do and what barriers prevent 
or discourage buying. Underhill describes himself 
as a “commercial researcher, which means | am 
part scientist, part artist, and part entrepreneur.” 
His company, Envirosell, has offices in the United 
States, Milan, Sidney, and Sao Paulo. Envirosell 
concentrates on the third segment of retail informa- 
tion, drawn from observation (segment 1 is register 
data, and segment 2 is communication studies). 
In an ABC News live e-chat, Underhill said, “The 
principal differences in 1st world shopping patterns 
are governed more by education and income than 
by ethnicity . . . but the Brits and Aussies [do] tend 
to walk as they drive. This sets up some very pecu- 
liar retail [shopping] patterns, because their walking If you were Gap and about to design a store to open in London, how would 
patterns set up a left-hand dominance, whereas you design a study to verify Paco Underhill’s conclusions about left-hand 
in the U.S. and much of the rest of the world, our dominance? 

walking patterns set up a right-hand dominance.” 


www.envirosell.com 


Predictions 


If we wanted to predict the price of a case of investment-grade red wine for a growing season that aver- 
ages 21°C, our prediction would be 


Y = —645.57 + 216.44(21) = 3,899.67 


This is a point prediction of Y and should be corrected for greater precision. As with other confidence 
estimates, we establish the degree of confidence desired and substitute into the formula 


a ee ieee 
= Ly 10 SS. 


where 


ty. = the two-tailed critical value for t at the desired level (95 percent in this example) 


s = the standard error of estimate (also the square root of the mean square error from the analy- 


sis of variance of the regression model) (see Exhibit 18-19). 
SS = the sum of squares for X (see Exhibit 18-13). 
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1. 21-1961" 
3,899.67 + (2.306)(538.559) V 1+ 35+ “a5E55 


3,899.67 + 1,308.29 


We are 95 percent confident of our prediction that a case of investment-quality red wine grown in a par- 
ticular year at 21°C average temperatures will be initially priced at 3,899.67 + 1,308.29 euros, or from 
approximately 2,591 to 5,208 EUR. The comparatively large bandwidth results from the amount of 
error in the model (reflected by 7°), some peculiarities in the Y values, and the use of a single predictor. 

It is more likely that we would want to predict the average price of all cases grown at 21°C. This 
prediction would use the same basic formula but omitting the first digit (the 1) under the radical. A nar- 
rower confidence band is the result since the average of all Y values is being predicted from a given X. In 
our example, the confidence interval for 95 percent is 3,899.67 +411.42, or from 3,488 to 4,311 EUR. 

The predictor we selected, 21°C, was close to the mean of X (19.61). Because the prediction and 
confidence bands are shaped like a bow tie, predictors farther from the mean have larger bandwidths. 
For example, X values of 15, 20, and 25 produce confidence bands of +565, +397, and +617, respec- 
tively. This is illustrated in Exhibit 18-17. The farther one’s selected predictor is from X, the wider is 
the prediction interval. 


Testing the Goodness of Fit 


With the regression line plotted and a few illustrative predictions, we should now gather some evi- 
dence of goodness of fit—how well the model fits the data. The most important test in bivariate linear 
regression is whether the slope, B,, is equal to zero.'' We have already observed a slope of zero in Ex- 
hibit 18-10, line b. Zero slopes result from various conditions: 

e Yis completely unrelated to X, and no systematic pattern is evident. 

¢ There are constant values of Y for every value of X. 


¢ The data are related but represented by a nonlinear function. 


>Exhibit 18-17 Prediction and Confidence Bands on Proximity to X 
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Y= Bo + ByX; 
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The Test 


To test whether B, = 0, we use a two-tailed test (since the actual relationship is positive, negative, or 
zero). The test follows the ¢ distribution for n — 2 degrees of freedom: 


— db, _ 216.439 _ 
ie, sao 


where 


b, was previously defined as the slope B, 


s(b,) is the standard error of B,'” 


We reject the null hypothesis, 8, = 0, because the calculated fis greater than any f value for 8 degrees 
of freedom and « = .01. Therefore, we conclude that the slope is not equal to zero. 


The F Test 


Computer printouts generally contain an analysis of variance (ANOVA) table with an F test of the 
regression model. In bivariate regression, t and F tests produce the same results since f° is equal to F. 
In multiple regression, the F test has an overall role for the model, and each of the independent vari- 
ables is evaluated with a separate t-test. From the last chapter, recall that ANOVA partitions variance 
into component parts. For regression, it comprises explained deviations, Y—  Y, and unexplained 
deviations, Y— Y. Together they constitute the total deviation, Y— _ Y. This is shown graphically in 
Exhibit 18-18. These sources of deviation are squared for all observations and summed across the 
data points. 

In Exhibit 18-19, we develop this concept sequentially, concluding with the F test of the regres- 
sion model for the wine data. Based on the results presented in that table, we find statistical evidence 
of a linear relationship between variables. The null hypothesis, 7? = 0, is rejected with F = 32.02, 
d.f. (1, 8), p < .005. The alternative hypothesis is accepted. The null hypothesis for the F test had the 
same effect as B, = O since we could select either test. Thus, we conclude that X and Y are linearly 
related. 


>Exhibit 18-18 Components of Variation 
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>Exhibit 18-19 Progressive Application of Partitioned Variance Concept 


(Y-Y) 


Explained Variation 
(the regression relationship 
between X and Y) 


I 


n a = 
= Voy 
=| 


SS; 
Sum of Squares Regression 


Source 


Regression 1 


Error n-2 


Total 


Source 
Regression 1 
Residual (error) 8 
Total 


Contents of Summary Table 


Degrees of Freedom Sum of Squares 


ANOVA Summary Table: Test of Regression Model 


Degrees of Freedom Sum of Squares 


General Concept 


(Y-Y) (Y-Y) 


Unexplained Variation Total Variation 
(cannot be explained by 


the regression relationship) 


ANOVA Application 


I 


z (Y-y)2 E(Y-V2 
= 


i=1 


SSp SSt 
Sum of Squares Error Sum of Squares Total 


Mean Square 


Ss. MS, = a we 
e 
SS. 
SS. MS, = —~£ 


Mean Square F Ratio 


9,287,143.11 9,287,143.11 32.02 
2,320,368.49 290,046.06 
11,607,511.60 


Significance of F = .0005 


Coefficient of Determination 


In predicting the values of Y without any knowledge of X, our best estimate would be Y, its mean. Each 


predicted value that does not fall on Y contributes to an error of estimate, Y — Y. The total squared 


error for several predictions would be 2(Y, — Y)’. By introducing known values of X into a regression 


equation, we attempt to reduce this error even further. Naturally, this is an improvement over using Y, 


and the result is (Y — Y). The total improvement based on several estimates is ee — Y)?, the amount 


of variation explained by the relationship between X and Y in the regression. Based on the formula, the 
coefficient of determination is the ratio of the line of best fit’s error over that incurred by using Y. One 
purpose of testing, then, is to discover whether the regression equation is a more effective predictive 
device than the mean of the dependent variable. 
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As in correlation, the coefficient of determination is symbolized by 7.'° It has several purposes. As 
an index of fit, it is interpreted as the total proportion of variance in Y explained by X. As a measure of 
linear relationship, it tells us how well the regression line fits the data. It is also an important indicator 
of the predictive accuracy of the equation. Typically, we would like to have an r° that explains 80 per- 
cent or more of the variation. Lower than that, predictive accuracy begins to fall off. The coefficient of 
determination, 7’, is calculated like this: 


>» -¥9 
i=1 
For the wine price study, r’ was found by using the data from the bottom of Exhibit 18-19: 


2,320,368.49 
11,607,511.60 


r=1 = .80 


Eighty percent of the variance in price may be explained by growing-season temperatures. With actual 
data and multiple predictors, our results would improve substantially. 


> Nonparametric Measures of Association” 


Measures for Nominal Data 


Nominal measures are used to assess the strength of relationships in cross-classification tables. They 
are often used with chi-square or may be used separately. In this section, we provide examples of three 
statistics based on chi-square and two that follow the proportional reduction in error approach. 

There is no fully satisfactory all-purpose measure for categorical data. Some are adversely affected 
by table shape and number of cells; others are sensitive to sample size or marginals. It is perturbing to 
find similar statistics reporting different coefficients for the same data. This occurs because of a statis- 
tic’s particular sensitivity or the way it was devised. 

Technically, we would like to find two characteristics with nominal measures: 


e When there is no relationship at all, the coefficient should be 0. 


e When there is a complete dependency, the coefficient should display unity, or 1. 


This does not always happen. In addition to being aware of the sensitivity problem, analysts should be 
alert to the need for careful selection of tests. 


Chi-Square-Based Measures 


Exhibit 18-20 reports a 2 x 2 table showing the test of an advertising campaign involving 66 people. 
The variables are success of the campaign and whether direct mail was used. In this example, the ob- 
served significance level is less than the testing level (a = .05), and the null hypothesis is rejected. A 
correction to chi-square is provided. We now turn to measures of association to detect the strength of 
the relationship. Notice that the exhibit also provides an approximate significance of the coefficient 
based on the chi-square distribution. This is a test of the null hypothesis that no relationship exists 
between the variables of direct mail and campaign success. 

The first chi-square-based measure is applied to direct mail and campaign success. It is called 
phi (db). Phi ranges from 0 to + 1.0 and attempts to correct x’ proportionately to NV. Phi is best employed 
with 2 x 2 tables like Exhibit 18-20 since its coefficient can exceed + 1.0 when applied to larger tables. 


Phi is calculated 
_4/X° _ 4. [6.616257 _ 
o=VF=\) a 3056 
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>Exhibit 18-20 Chi-Square-Based Measures of Association 


Marketing Campaign Success 
Count Row 
Yes No Total 
Yes 21 10 31 
Direct Mail 
No 13 22 35 
Column 
Total 34 32 66 
Chi-Square Value d.f. Significance 
Pearson 6.16257 1 .01305 
Continuity correction 4.99836 1 .02537 
Minimal expected frequency 15.030 
Approximate 
Statistic Value Significance 
Phi .30557 .01305° 
Cramer’s V .30557 .01305" 
Contingency coefficient C -29223 .01305" 
“Pearson chi-square probability. 


Phi’s coefficient shows a moderate relationship between marketing campaign success and direct mail. 
There is no suggestion in this interpretation that one variable causes the other, nor is there an indication 
of the direction of the relationship. 

Cramer’s V is a modification of phi for larger tables and has a range up to 1.0 for tables of any 
shape. It is calculated like this: 


a X= /6.616257 — 
V Vt 4/ 66(D 3056 


where k = the lesser number of rows or columns. In Exhibit 18-20, the coefficient is the same as phi. 
The contingency coefficient C is reported last. It is not comparable to other measures and has a dif- 
ferent upper limit for various table sizes. The upper limits are determined as 


k-1 
k 
where k = the number of columns. For a 2 X 2 table, the upper limit is .71; fora 3 X 3, .82; and for 


a4 X 4, .87. Although this statistic operates well with tables having the same number of rows as col- 
umns, its upper-limit restriction is not consistent with a criterion of good association measurement. 


Cis calculated as 
Xx _4/ 6.616257 _ 
YV+N 6.616257 + 66 e2ee 


The chief advantage of C is its ability to accommodate data in almost every form: skewed or normal, 
discrete or continuous, and nominal or ordinal. 
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Proportional Reduction in Error 


Proportional reduction in error (PRE) statistics are the second type used with contingency tables. 
Lambda and tau are the examples discussed here. The coefficient lambda (A) is based on how well 
the frequencies of one nominal variable offer predictive evidence about the frequencies of another. 
Lambda is asymmetrical—allowing calculation for the direction of prediction—and symmetrical, pre- 
dicting row and column variables equally. 

The computation of lambda is straightforward. In Exhibit 18-21, we have results from an opin- 
ion survey with a sample of 400 shareholders in publicly traded firms. Of the 400 shareholders, 
180 (45 percent) favor capping executives’ salaries; 220 (55 percent) do not favor doing so. With this 
information alone, if asked to predict the opinions of an individual in the sample, we would achieve 
the best prediction record by always choosing the modal category. Here it is “do not favor.” By doing 
so, however, we would be wrong 180 out of 400 times. The probability estimate for an incorrect clas- 
sification is .45, P(1) = (1 — .55). 

Now suppose we have prior information about the respondents’ occupational status and are asked 
to predict opinion. Would it improve predictive ability? Yes, we would make the predictions by sum- 
ming the probabilities of all cells that are not the modal value for their rows [for example, cell (1, 2) is 
20/400, or .05]: 


P(2) = cell (1, 2) .05 + cell (2, 1) .15 + cell (3, 1) .075 = .275 


>Exhibit 18-21 Proportional Reduction of Error Measures 


What is your opinion about capping executives’ salaries? 


Cell designation Favor Do Not Row 
Count Favor Total 
Row Pct. 
Managerial 1,1 Hea 110 
20 
82.0 18.0 
Occupational White collar Ea 2,2 140 
Class 60 80 
43.0 57.0 
Blue collar | Sil | 3,2 150 
30 120 
20.0 80.0 
Column 180 — 220 400 — 
Total 45.0% 55.0% 100.0% 
Chi-Square Value d.f. Significance 
Pearson 98.38646 2 .00000 
Likelihood ratio 104.96542 2 .00000 


Minimum expected frequency 49.500 


Approximate 

Statistic Value ASEI T Value Significance 
Lambda: 

Symmetric .30233 .03955 6.77902 

With occupation dependent .03820 5.69495 

With opinion dependent .04555 7.08010 
Goodman & Kruskal tau: 

With occupation dependent -11669 .02076 .00000* 

With opinion dependent .24597 .03979 .00000* 


“Based on chi-square approximation. 
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Lambda is then calculated: 


_ PQ) ~ PQ) _ 45-275 _ 


A P() 45 


Note that the asymmetric lambda in Exhibit 18-21, where opinion is the dependent variable, reflects 
this computation. As a result of knowing the respondents’ occupational classification, we improve our 
prediction by 39 percent. If we wish to predict occupational classification from opinion instead of the 
opposite, a \ of .24 would be secured. This means that 24 percent of the error in predicting occupational 
class is eliminated by knowledge of opinion on the executives’ salary question. Lambda varies between 
0 and 1, corresponding with no ability to eliminate errors to elimination of all errors of prediction. 

Goodman and Kruskal’s tau (tT) uses table marginals to reduce prediction errors. In predicting opin- 
ion on executives’ salaries without any knowledge of occupational class, we would expect a 50.5 
percent correct classification and a 49.5 percent probability of error. These are based on the column 
marginal percentages in Exhibit 18-21. 


Column Marginal Column Percent Correct Cases 
180 ae 45 = 81 
220 : 5) = Zi 
Total correct classification 202 
202 


Correct classification of the opinion variable = .505 = 400 


Probability of error, P(1) = (1 — .505) = .495 


When additional knowledge of occupational class is used, information for correct classification of 
the opinion variable is improved to 62.7 percent with a 37.3 percent probability of error. This is obtained 
by using the cell counts and marginals for occupational class (refer to Exhibit 18-21), as shown below: 


Row 1 [20 90 [AR)20 = 73.6364+3.6864 = 77.2727 
Row 2 | al 60 ($3) 80 = 257148245142 = 71.4286 
Row 3 | al 30 (323) 120 = 60+96.0 = 102.0000 


Total correct classification (with additional information on occupational class) 250.7013 


Correct classification of the opinion variable = .627 = st 


Probability of error, P(2) = (1 — .627) = .373 


Tau is then computed like this: 


_ PU) — PQ) _ 495 — 373 _ 
: Pl) 495 a 


Exhibit 18-21 shows that the information about occupational class has reduced error in predicting 
opinion to approximately 25 percent. The table also contains information on the test of the null 
hypothesis that tau = 0 with an approximate observed significance level and asymptotic error (for 
developing confidence intervals). Based on the small observed significance level, we would conclude 
that tau is significantly different from a coefficient of 0 and that there is an association between 
opinion on executives’ salaries and occupational class in the population from which the sample was 
selected. We can also establish the confidence level for the coefficient at the 95 percent level as 
approximately .25 = .04. 
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Advanced Statistics Increase Satisfaction and Release More Funds through ATMs 


The Navy Federal Credit Union (NFCU) is the world’s larg- 
est credit union, with more than $48 billion in assets, 3.9 mil- 
lion members, 220 branch offices, 461 proprietary ATMs, and 
more than 8,900 employees worldwide. Its clientele represent 
all Department of Defense military and civilians and their families. 
According to Alan Payne, Navy’s manager of member research 
and development, it regularly surveys members’ satisfaction with 
lending, savings and checking, and investments and insurance 
programs using various products from SPSS PASW statistical, 
modeling, and text analysis modules. Through a combination 
of statistical techniques NFCU discovered that members phon- 
ing the call center for account information also wanted to know 
about enhanced services and were receptive to cross-servicing 
promotions. Payne added that by discovering voice-of-customer 
insights on satisfaction levels, NFCU realized 15.3 times return 
on technological investment over a two-month period, creating 
almost $1.5 million benefit annually. 

Boeing Employees’ Credit Union (BECU) is another proac- 
tive organization using sophisticated analytics. As a top finan- 
cial cooperative in the United States with more than 744,000 
members, two full-service centers, and more than 40 locations, 


Boeing recently launched an ambitious companywide research 
project to improve customer satisfaction. The project focused on 
whether to allow larger ATM withdrawals and more provisional 
credit for checks deposited. BECU hypothesized that allowing 
members to access cash more conveniently would improve 
retention and generate revenue. Additional objectives were to 
optimize response rates in direct marketing campaigns, lower 
per-unit acquisition costs, and identify new branch locations. 

Calvin Bierley, market research analyst, in talking about its 
use of SPSS said, “A risk score model was embedded in the 
daily transactions processing system to automatically determine 
how much cash each member can withdraw from an ATM or 
receive when making deposits.” Then through other modeling 
statistical techniques, BECU identified receptive customers for 
marketing campaigns (achieving a 20 to 30 percent response 
rate on direct mail), saved $1 million annually in staffing by au- 
tomating provisional credit decisions, increased revenue by 
$600,000 from new member acquisition and retention, and suc- 
cessfully identified new branch locations. 


www.becu.org; www.navyfcu.org; www.spss.com 


Measures for Ordinal Data 


When data require ordinal measures, there are several statistical alternatives. In this section we will 


illustrate: 


e Gamma. 
e Kendall’s tau b and tau c. 
e Somers’s d. 


e Spearman’s rho. 


All but Spearman’s rank-order correlation are based on the concept of concordant and discordant pairs. 
None of these statistics require the assumption of a bivariate normal distribution, yet by incorporating 
order, most produce a range from —1.0 (a perfect negative relationship) to +1.0 (a perfect positive 
one). Within this range, a coefficient with a larger magnitude (absolute value of the measure) is inter- 
preted as having a stronger relationship. These characteristics allow the analyst to interpret both the 
direction and the strength of the relationship. 

Exhibit 18-22 presents data for 70 managerial employees of KeyDesign, a large industrial design 
firm. All 70 employees have been evaluated for coronary risk by the firm’s health insurer. The manage- 
ment levels are ranked, as are the fitness assessments by the physicians. If we were to use a nominal 
measure of association with these data (such as Cramer’s V), the computed value of the statistic would 
be positive since order is not present in nominal data. But using ordinal measures of association reveals 
the actual nature of the relationship. In this example, all coefficients have negative signs; therefore, 
lower levels of fitness are associated with higher management levels. 
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>Exhibit 18-22 Tabled Ranks for Management and Fitness Levels at KeyDesign 


Management Level 
Count Lower Middle Upper 
High 14 4 2 20 
Fitness Moderate 18 6 2 26 
Low 2 6 16 24 
34 16 20 70 
Statistic Value” 
Gamma | = 
Kendall’s tau 6b -51 
Kendall’s tau c 
Somers’s d 
Symmetric -.51 
With fitness dependent -.53 
With management-level dependent -.50 
“The t value for each coefficient is -5.86451. 


The information in the exhibit has been arranged so that the number of concordant and discordant 
pairs of individual observations may be calculated. When a subject that ranks higher on one variable 
also ranks higher on the other variable, the pairs of observations are said to be concordant. If a higher 
ranking on one variable is accompanied by a lower ranking on the other variable, the pairs of observa- 
tions are discordant. Let P stand for concordant pairs and Q stand for discordant. When concordant 
pairs exceed discordant pairs in a P — Q relationship, the statistic reports a positive association between 
the variables under study. As discordant pairs increase over concordant pairs, the association becomes 
negative. A balance indicates no relationship between the variables. Exhibit 18-23 summarizes the 
procedure for calculating the summary terms needed in all the statistics we are about to discuss.'* 

Goodman and Kruskal’s gamma (y) is a statistic that compares concordant and discordant pairs and 
then standardizes the outcome by maximizing the value of the denominator. It has a proportional reduc- 
tion in error (PRE) interpretation that connects nicely with what we already know about PRE nominal 
measures. Gamma is defined as 


_P—-Q _ 172-992 _ —820 _ 
Y"P+OQ 1724992 1164 270) 


For the fitness data, we conclude that as management level increases, fitness decreases. This is im- 
mediately apparent from the larger number of discordant pairs. A more precise explanation for gamma 
takes its absolute value (ignoring the sign) and relates it to PRE. Hypothetically, if one was trying to 
predict whether the pairs were concordant or discordant, one might flip a coin and classify the outcome. 
A better way is to make the prediction based on the preponderance of concordance or discordance; the 
absolute value of gamma is the proportional reduction in error when prediction is done the second way. 
For example, you would get a 50 percent hit ratio using the coin. A PRE of .70 improves your hit ratio 
to 85 percent (.50 X .70) + (.50) = .85. 

With a y of —.70, 85 percent of the pairs are discordant and 15 percent are concordant.'® There are 
almost six times as many discordant pairs as concordant pairs. In situations where the data call for a 
2 X 2 table, the appropriate modification of gamma is Yule’s Q.'’ 

Kendall’s tau b (7,) is a refinement of gamma that considers tied pairs. A tied pair occurs when 
subjects have the same value on the X variable, on the Y variable, or on both. For a given sample size, 
there are n(n — 1)/2 pairs of observations.'* After concordant pairs and discordant pairs are removed, 
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>Exhibit 18-23 Calculation of Concordant (P), Discordant (Q), Tied (Tal), and Total Paired Observations: 


KeyDesign Example 


Management 
Lower Middle Upper 
n 
Total 3% 
pairs £ n(n - 1)/2 = 70(69)/2 = 2,415 
Le 
Concordant 
pairs 
172 
Discordant 
pairs 
1464+ 2+6+16) + 4(2 + 18) + 992 
20 
; r m(m,-1) 20(19) 26(25) 24(23) 
Tied Ter 1 - = 791 
pairs 26 ey a 2 
Total tied 
24 fitness 
(m,—-1 16(1 20(1 
34 16 20 70 T= mnt) = a + es) + pall = 871 
= 2 2 2 2 , 
Total tied 
management 
where _T, is the total pairs of ties on the column variable 
Ty is the total pairs of ties on the row variable 
m,, are the marginals 


the remainder are tied. Tau b does not have a PRE interpretation but does provide a range of + 1.0 to 
—1.0 for square tables. Its compensation for ties uses the information found in Exhibit 18-23. It may 


be calculated as 
P-@Q 


1 Ves D.. es Dip 


_ 172 — 992 =-5] 


 V/QA15 — 87-)2,415 — 791) 


>chapter 18 Measures of Association 


Kendall’s tau c (7,) is another adjustment to the basic P — Q relationship of gamma. This approach 
to ordinal association is suitable for tables of any size. Although we illustrate tau c, we would select 
tau b since the cross-classification table for the fitness data is square. The adjustment for table shape is 
seen in the formula 


Im(P — Q) 2(3)(172 — 992) 
Te N(m—1) (703 — 1) 


where m is the smaller number of rows or columns. 

Somers’s d rounds out our coverage of statistics employing the concept of concordant- 
discordant pairs. This statistic’s utility comes from its ability to compensate for tied ranks and 
adjust for the direction of the dependent variable. Again, we refer to the preliminary calculations 
provided in Exhibit 18-23 to compute the symmetric and asymmetric d’s. As before, the symmet- 
ric coefficient (equation 1) takes the row and column variables into account equally. The second 
and third calculations show fitness as the dependent and management level as the dependent, 
respectively. 


(P — Q) —820 _ 
oes ~ a(n — 1) - QT /2 ~ 7,584 — — S51 () 
= (P-—Q) _ —820  _ 
d,_, n(n — 1) 2,415 — 871 oe (2) 
2 x 
(f=) —820. _ 
xy n(n — 1) _ T 2,415 — 791 = a 
2 y 


The Spearman’s rho (p) correlation is another ordinal measure. Along with Kendall’s tau, it is used 
frequently with ordinal data. Rho correlates ranks between two ordered variables. Occasionally, 
researchers find continuous variables with too many abnormalities to correct. Then scores may be 
reduced to ranks and calculated with Spearman’s rho. 

As a special form of Pearson’s product moment correlation, rho’s strengths outweigh its weak- 
nesses. First, when data are transformed by logs or squaring, rho remains unaffected. Second, outliers 
or extreme scores that were troublesome before ranking no longer pose a threat since the largest num- 
ber in the distribution is equal to the sample size. Third, it is an easy statistic to compute. The major 
deficiency is its sensitivity to tied ranks. Ties distort the coefficient’s size. However, there are rarely 
too many ties to justify the correction formulas available. 

To illustrate the use of rho, consider a situation where KDL, a media firm, is recruiting account ex- 
ecutive trainees. Assume the field has been narrowed to 10 applicants for final evaluation. They arrive 
at the company headquarters, go through a battery of tests, and are interviewed by a panel of three ex- 
ecutives. The test results are evaluated by an industrial psychologist who then ranks the 10 candidates. 
The executives produce a composite ranking based on the interviews. Your task is to decide how well 
these two sets of ranking agree. Exhibit 18-24 contains the data and preliminary calculations. Substitut- 
ing into the equation, we get 


6x2 _ __6(57) 


~=n Gor—=0 


r,=1 


where 7 is the number of subjects being ranked. 

The relationship between the panel’s and the psychologist’s rankings is moderately high, suggest- 
ing agreement between the two measures. The test of the null hypothesis that there is no relationship 
between the measures (r, = 0) is rejected at the .05 level with n — 2 degrees of freedom. 


= n-2 _ 8 = 
: cae —— ane 
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>Exhibit 18-24 KDL Data for Spearman’s Rho 


Rank by 


Applicant Panel x 


Psychologist y 


= 


3.5 
10.0 
6.5 
210) 
1.0 
9.0 
3.5 
6.5 
8.0 
5.0 


Oo GO AN Oak WwW DY 


= 


6.0 =29 6.25 
5.0 5.0 25.00 
8.0 = 1105 2.25 
1.5 0.5 0.25 
3.0 =20) 4.00 
a0) 2.0 4.00 
1.5 2.0 4.00 
9.0 = 25) 6.25 
10.0 =210 4.00 
4.0 1.0 1.00 
57.00 


Note: Tied ranks were assigned the average (of ranks) as if no ties had occurred. 


Management questions frequently involve relationships be- 
tween two or more variables. Correlation analysis may be 
applied to study such relationships. A correct correlational 
hypothesis states that the variables occur together in some 
specified manner without implying that one causes the other. 


Parametric correlation requires two continuous variables 
measured on an interval or ratio scale. The product moment 
correlation coefficient represents an index of the magnitude 
of the relationship: Its sign governs the direction and its 
square explains the common variance. Bivariate correlation 
treats X and Y variables symmetrically and is intended for 
use with variables that are linearly related. 

Scatterplots allow the researcher to visually inspect re- 
lationship data for appropriateness of the selected statistic. 
The direction, magnitude, and shape of a relationship are 
conveyed in a plot. The shape of linear relationships is char- 
acterized by a straight line, whereas nonlinear relationships 
are curvilinear or parabolic or have other curvature. The as- 
sumptions of linearity and bivariate normal distribution may 
be checked through plots and diagnostic tests. 

A correlation coefficient of any magnitude or sign, re- 
gardless of statistical significance, does not imply causation. 
Similarly, a coefficient is not remarkable simply because it is 
statistically significant. Practical significance should be con 
sidered in interpreting and reporting findings. 


Regression analysis is used to further our insight into the re- 
lationship of Y with X. When we take the observed values o 
X to estimate or predict corresponding Y values, the process 
is called simple prediction. When more than one X variable is 
used, the outcome is a function of multiple predictors. Simple 
and multiple predictions are made with regression analysis. 


A straight line is fundamentally the best way to model 
the relationship between two continuous variables. The 
method of least squares allows us to find a regression line, 
or line of best fit, that minimizes errors in drawing the line. 
It uses the criterion of minimizing the total squared errors 
of estimate. Point predictions made from well-fitted data 
are subject to error. Prediction and confidence bands may 
be used to find a range of probable values for Y based 
on the chosen predictor. The bands are shaped in such 
a way that predictors farther from the mean have larger 
bandwidths. 


We test regression models for linearity and to discover 
whether the equation is effective in fitting the data. An 
important test in bivariate linear regression is whether the 
slope is equal to zero (i.e., whether the predictor variable X 
is a significant influence on the criterion variable Y). In bivari- 
ate regression, t-tests and F tests of the regression produce 
he same result since ¢? is equal to F. 


Often the assumptions or the required measurement level 
or parametric techniques cannot be met. Nonparametric 
measures of association offer alternatives. Nominal mea- 
sures of association are used to assess the strength of 
relationships in cross-classification tables. They are often 
used in conjunction with chi-square or may be based on the 
proportional reduction in error (PRE) approach. 

Phi ranges from 0 to +1.0 and attempts to correct chi- 
square proportionately to NV. Phi is best employed with 2 x 
2 tables. Cramer's V is a modification of phi for larger tables 
and has a range up to 1.0 for tables of any configuration. 
Lambda, a PRE statistic, is based on how well the frequen- 
cies of one nominal variable offer predictive evidence about 


the frequencies of another. Goodman and Kruskal’s tau uses 
table marginals to reduce prediction errors. 
Measures for ordinal data include gamma, Kendall’s 
tau b and tau c, Somers’s d, and Spearman’s rho. All 
but Spearman’s rank-order correlation are based on the 


Be 


artifact correlations 476 
bivariate correlation analysis 469 
bivariate normal distribution 473 
chi-square-based measures 490 
contingency coefficient C 491 
Cramer’s V 491 
phi () 490 
coefficient of determination (r?) 474 
concordant 495 
discordant 495 


error term 482 


Terms in Review 
1 Distinguish between the following: 


goodness of fit 487 
lambda (A) 492 
linearity 473 


method of least squares 482 


ordinal measures 494 
gamma (y) 495 
Somers’s d 497 


Spearman’s rho (p) 497 


tau (t) 493 
tau b(7,) 495 
tau c(z,) 497 


a Regression coefficient and correlation coefficient. 


b r=Oandp=0. 


c. The test of the true slope, the test of the intercept, and 


P=0. 
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concept of concordant and discordant pairs. None of these 
statistics require the assumption of a bivariate normal dis- 
tribution, yet by incorporating order, most produce a range 
from —1 to +1. 


Pearson correlation coefficient 469 
prediction and confidence bands 487 


proportional reduction in error 
(PRE) 492 


regression analysis 479 
regression coefficients 479 
intercept (8,) 480 
slope (8,) 479 
residual 485 
scatterplot 470 
simple prediction 479 


d randr. 
e Aslope of 0. 
f Fandt??. 


2 Describe the relationship between the two variables in the 


four plots. 


(d) 
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Making Research Decisions 
3 A polling organization collected data on a sample of 60 
registered voters regarding a tax on the market value of 
equity transactions as one remedy for the budget deficit. 


Education 


Opinion about 


Market Tax High School College Grad. MBA 
Favorable 15 5 ) 
Undecided 10 8 2 
Unfavorable ) 2 18 


a Compute gamma for the table. 

b Compute tau 6 or tau c for the same data. 

c What accounts for the differences? 

d Decide which is more suitable for these data. 


4 Using the table data in question 3, compute Somers’s d 
symmetric and then use opinion as the dependent variable. 
Decide which approach is best for reporting the decision. 


5 Aresearch team conducted a study of soft-drink prefer- 
ences among residents in a test market prior to an adver- 
tising campaign for a new cola product. Of the participants, 
130 are teenagers and 130 are adults. The researchers 
secured the following results: 


Cola Noncola 
Teenagers 50 80 
Adults 90 40 


Calculate an appropriate measure of association, and de- 
cide how to present the results. How might this information 
affect the advertising strategy? 


Bringing Research to Life 
6 What would the numbers of “police calls resulting in arrest” 
for Gladeside and Oceanside need to change to in order to 
support the conclusion of “disparate impact.” 


From Concept to Practice 
7 Using the following data, 


X ¥. 
3 6 
6 10 
9 15 
12 24 
15 21 


a Create a scatterplot. 
b Find the least-squares line. 
c Plot the line on the diagram. 


d Predict: Y if X is 10. 
YifXis 17. 


8 Ahome pregnancy test claims to be 97 percent accurate 
when consumers obtain a positive result. To what extent 
are the variables of “actual clinical condition” and “test 
readings” related? 


a Compute phi, Cramer’s V, and the contingency coeffi- 
cient for the following table. What can you say about the 
strength of the relationship between the two variables? 


b Compute lambda for these data. What does this statis- 
tic tell you? 
Actual Clinical Condition Test Readings of In-Vitro 
Diagnostic Cross-Tabulation 
Test Readings of 


Count In-Vitro Diagnostic 

Actual Clinical 

Condition Positive Negative Total 

Pregnant 451 36 487 
accurate inaccurate 

Not pregnant 15 183 198 
inaccurate accurate 

Total 466 219 685 


9 Fill in the missing blocks for the ANOVA summary table on 
net profits and market value used with regression analysis. 


ANOVA Summary Table 
Sum of Mean 
d.f. Squares Square F 
Regression 1 11,116,995.47 
Error 116,104.63 


Total 9 12,045,832.50 


a What does the F tell you? (alpha = .05) 
b What is the t value? Explain its meaning. 


(See table on next page for data.) 
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Forbes 500 Random Subsample ($, millions) 

Number of 

Employees 

Assets Sales Market Value Net Profit Cash Flow (thousands) 
1,034.00 1,510.00 697.00 82.60 126.50 16.60 
956.00 785.00 1,271.00 89.00 191.20 5.00 
1,890.00 2,533.00 1,783.00 176.00 267.00 44.00 
1,133.00 582.00 752.00 82.30 137.10 2.10 
11,682.00 3,790.00 4,149.00 413.50 806.80 11.90 
6,080.00 635.00 291.00 18.10 35.20 3.70 
31,044.00 3,296.00 2,705.00 337.30 425.50 20.10 
5,878.00 3,204.00 2,100.00 145.80 380.00 10.80 
1,721.00 981.00 1,573.00 172.60 326.60 1.90 
2,135.00 2,268.00 2,634.00 247.20 355.50 21.20 


10 Secure Spearman rank-order correlations for the largest (1) 6, =0. 
Pearson coefficient in the matrix from question 9. Explain (2) r=0. 
the differences between the two findings. 


(3) An F test. 


11 Using the preceding matrix data (Forbes 500), select a pair ae ee ee 


of variables and run a simple regression. Then investigate 

the appropriateness of the model for the data using diag- 13 “There is an ‘unhealthy correlation’ between the building of 
nostic tools for evaluating assumptions. skyscrapers and subsequent financial crashes,” according 
to Barclays Capital. Examples include: the world’s first sky- 
scraper, the Equitable Life building in New York (completed 
xX Y in 1873 and coincided with a five-year recession), the 
Empire State building (the Great Depression was under 


12 For the following data, 


= way), Chicago’s Willis Tower—formerly known as the Sears 
i Tower—in 1974 (during an oil shock and when the U.S. 
W 12 dollar’s reliance on gold was abandoned), and Malaysia’s 
14 23 Petronas Towers in 1997 (corresponding to the Asian 
12 20 financial crisis). Currently, the world’s tallest skyscraper is 
9 25 the Burj Khalifa (built just before Dubai’s financial troubles). 
8 26 China is the biggest builder of skyscrapers, with 53 percent 
- 28 of all the tall buildings in the world. JPMorgan Chase said 
that the Chinese property market could drop by as much 
2 as 20 percent in value in the country’s major cities within 
the next 12 to 18 months. “Often the world’s tallest build- 
a Calculate the correlation between X and Y. ings are . . . [a reflection of] a widespread misallocation of 
b Interpret the sign of the correlation. capital and an impending economic correction,” Barclays 


c_ Interpret the square of the correlation. Capital analysts said. 


Examine the correlational finding and provide plausible 
alternatives for their findings. 


d Plot the least-squares line. 


e Test for a linear relationship: 


Fe 
Mastering Teacher Leadership NCRCC: Teeing Up and New Strategic Direction 
* You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine whether a 


case provides data, the research instrument, video, or other supplementary material. Written cases are downloadable from the 
text website (www.mhhe.com/cooper1 2e). All video material and video cases are available from the Online Learning Center. 
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learningobjectives 
After reading this chapter, you should understand . . . 


That a quality presentation of research findings can have an inordinate effect on a reader’s or a listener’s perceptions 
of a study’s quality. 


The contents, types, lengths, and technical specifications of research reports. 


That the writer of a research report should be guided by questions of purpose, readership, circumstances/limitations, 
and use. 


That while some statistical data may be incorporated in the text, most statistics should be placed in tables, charts, 
or graphs. 


6 6People are amazing at collecting data, but 
they’re often less skilled at creating insights 
out of it and spreading them throughout the 
whole organization. Data is great, but it rarely 


means anything unless you’ve figured out 
exactly what that data is saying—and what 
you’re going to do about it.9 9 


Nancy Porte, vice president of customer experience, 
Verint-Vovici 


bringingresearchtolife ...<° 


“Has it occurred to you that your draft of the MindWriter 
report has not been touched in the last two days? The 
stack of marked-up pages is right there on your desk, 
and you have been working around it.” 

Jason frowns and momentarily flicks his eyes to the 
stack of marked pages. 

Sara plunges ahead with her complaint. “It’s no 
big deal, you know. You promised to chop out three 
pages of methodology that nobody will care about 
but your fellow statistics jocks . . ..—Jason shoots 
her an aggrieved look— “. . . and to remove your 
recommendations and provide them in a separate, 
informal letter so that Myra Wines can distribute them 
under her name and claim credit for your ‘brilliance.’”’ 

“T think I have writer’s block.” 

“No. Writer’s block is when you can’t write. You 
can’t unwrite; that’s the problem. You have unwriter’s 
block. Look, some people do great research and then 
panic when they have to decide what goes in the report 
and what doesn’t. Or they can’t take all the great 
ideas running around in their heads and express their 
abstractions in words. Or they don’t believe they are 
smart enough to communicate with their clients, or 
vice versa. So they freeze up. This isn’t usually your 
problem. There is some sort of emotional link to this 
MindWriter report, Jason; face it.” 


“T love the MindWriter project.” 

“Ah, there’s the problem,” she says. “Jason, I have 
heard you say that you hate projects for other clients, 
and I have heard you say that you like projects. But this 
is the first time I have heard you say you love a project. 
There comes a time when, after you have nurtured 
something, you have to let go. Then it isn’t yours. It 
is someone else’s, or it is its own thing, but it is not 
yours.” 

“T guess you're right,” Jason smiles sheepishly. 
“Tm a little too invested. This MindWriter project 
was my baby—well, yours and mine. If I chop three 
pages out of the report, it is finished. Then it belongs 
to Myra. I don’t own it anymore. I can’t implement my 
recommendations. I can’t change anything. I can’t have 
second thoughts.” 

“Fix it, then. Send MindWriter an invoice. Write 
a proposal for follow-up work. Do something, Jason. 
Finish it. Let go and move on.” Sara smiles and pauses 
as she is about to leave Jason’s office. Jason has pulled 
the report to the center of his desk, a very good sign. 

“By the way, Custom Foods just called. It awarded 
us the contract for its ideation work. I’d hate to work on 
that project without you, but...” 
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> Introduction 


As part of the research proposal, the sponsor and the researcher agree on what types of reporting will 
occur both during and at the end of the research project. Depending on the budget for the project, a for- 
mal oral presentation may not be part of the reporting. A research sponsor, however, is sure to require 
a written report. Exhibit 19-1 details the reporting phase of the research process. 


> The Written Research Report 


It may seem unscientific and even unfair, but a poor final report or presentation can destroy a study. 
Research technicians may ignore the significance of badly reported content, but most readers will be 
influenced by the quality of the reporting. This fact should prompt researchers to make special efforts 
to communicate clearly and fully. 

The research report contains findings, analyses of findings, interpretations, conclusions, and some- 
times recommendations. The researcher is the expert on the topic and knows the specifics in a way no 
one else can. Because a written research report is an authoritative one-way communication, it imposes 
a special obligation for maintaining objectivity. Even if your findings seem to point to an action, you 
should demonstrate restraint and caution when proposing that course. 

Reports may be defined by their degree of formality and design. The formal report follows a well- 
delineated and relatively long format. This is in contrast to the informal or short report. 


Short Reports 


Short reports are appropriate when the problem is well defined, is of limited scope, and has a simple 
and straightforward methodology. Most informational, progress, and interim reports are of this kind: a 
report of cost-of-living changes for upcoming labor negotiations or an exploration of filing “dumping” 
charges against a foreign competitor. 

Short reports are about five pages. If used on a website, they may be even shorter. At the beginning, 
there should be a brief statement about the authorization for the study, the problem examined, and its 
breadth and depth. Next come the conclusions and recommendations, followed by the findings that 
support them. Section headings should be used. 

A letter of transmittal is a vehicle to convey short reports. A five-page report may be produced to 
track sales on a quarterly basis. The report should be direct, make ample use of graphics to show trends, 
and refer the reader to the research department for further information. Detailed information on the 
research method would be omitted, although an overview could appear in an appendix. The purpose of 
this type of report is to distribute information quickly in an easy-to-use format. Short reports are also 
produced for clients with small, relatively inexpensive research projects. 

The letter is a form of a short report. Its tone should be informal. The format follows that of any 
good business letter and should not exceed a few pages. A letter report is often written in personal style 
(we, you), although this depends on the situation. 

Memorandum reports are another variety and follow the To, From, Subject format. 

These suggestions may be helpful for writing short reports: 


Tell the reader why you are writing (it may be in response to a request). 


e If the memo is in response to a request for information, remind the reader of the exact point 
raised, answer it, and follow with any necessary details. 


e Write in an expository style with brevity and directness. 
° If time permits, write the report today and leave it for review tomorrow before sending it. 


e Attach detailed materials as appendices when needed. 
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>Exhibit 19-1 Written Presentation and the Research Process 


Determine Written Report 
Style & Type 


Determine Written Report Access 


Develop Appropriate Data 
Displays 


Prepare Nondata 
Sections of Report 


Compile Written Report 
Deliver Report 


Address Report 
Questions 


Long Reports 


Long reports are of two types, the technical or base report and the management report. The choice 
depends on the audience and the researcher’s objectives. 

Many projects will require both types of reports: a technical report, written for an audience of 
researchers, and a management report, written for the nontechnically oriented manager or client. 
Although some researchers try to write a single report that satisfies both needs, this complicates the 
task and is seldom satisfactory. The two types of audiences have different technical training, interests, 


and goals. 
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E-Speed or No Speed 


The Internet has forever transformed a decision maker’s expec- e Real-time frequencies. 

tations relating to speed of data results. The Internet has given e Real-time cross-tabulation. 

marketers a taste of “e-time”—data tabulated and synthesized e Real-time open-ended verbatim responses. 
in real time. “While some marketer’s realize that e-speed may e Real-time quota status. 

sacrifice quality in research,” shared Darcy Zwetko, manager— e Daily status of your projects. 

research services, Opinion Search, Inc. (Ottawa, Canada), e Daily call disposition reports. 


“knowing the speed available with online surveys encouraged us 
to make our CATI survey results accessible in real time, directly 
via each client’s computer.” Opinion Search calls this industry 
first dataCAP for “Data Control and Access Portal.” Among other 
opportunities, dataCAP provides: 


Besides being fast, dataCAP delivers encrypted information to pro- 
tect data privacy. Speed without safety won’t win a firm any clients. 
And speed that sacrifices quality won’t have them coming back. 


www.opinionsearch.com 


The Technical Report 


This report should include full documentation and detail. It will normally survive all working papers 
and original data files and so will become the major source document. It is the report that other re- 
searchers will want to see because it has the full story of what was done and how it was done. 

Although completeness is a goal, you must guard against including nonessential material. A good 
guide is that sufficient procedural information should be included to enable others to replicate the 
study. This includes sources of data, research procedures, sampling design, data gathering instru- 
ments, index construction, and data analysis methods. Most information should be attached in an 
appendix. 

A technical report should also include a full presentation and analysis of significant data. Conclu- 
sions and recommendations should be clearly related to specific findings. Technical jargon should be 
minimized but defined when used. There can be brief references to other research, theories, and tech- 
niques. Although you expect the reader to be familiar with these references, it is useful to include some 
short explanations, perhaps as footnotes or endnotes. 

The short technical report covers the same items as the long technical report but in an abbreviated 
form. The methodology is included as part of the introduction and takes no more than a few paragraphs. 
Most of the emphasis is placed on the findings and conclusions. A memo or letter format covers only 
the minimum: what the problem is and what the research conclusions are. 


The Management Report 


In contrast to the technical report, the management report is for the nontechnical client. The reader has 
little time to absorb details and needs a prompt exposure to the most critical findings; thus the report’s 
sections are in an inverted order. After the prefatory and introductory sections, the conclusions with 
accompanying recommendations are presented. Individual findings are presented next, supporting the 
conclusions already made. The appendices present any required methodological details. The order of 
the management report allows clients to grasp the conclusions and recommendations quickly, without 
much reading. Then, if they wish to go further, they may read on into the findings. The management 
report should make liberal use of visual displays. 

Sometimes the client has no research background and is interested in results rather than in method- 
ology. The major communication medium in this situation is the management report. It is still helpful 
to have a technical report if the client later wishes to have a technical appraisal of the study. 
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The style of the report should encourage rapid reading and quick comprehension of major findings, 
and it should prompt understanding of the implications and conclusions. The report tone is journal- 
istic and must be accurate. Headlines and underlining for emphasis are helpful; pictures and graphs 
often replace tables. Sentences and paragraphs should be short and direct. Consider liberal use of 
white space and wide margins. It may be desirable to put a single finding on each page. It also helps 
to have a theme running through the report and even graphic or animated characters designed to vary 
the presentation. 


> Research Report Components 


Research reports, long and short, have a set of identifiable components. Usually headings and subhead- 
ings divide the sections. Each report is individual; sections may be dropped or added, condensed or 
expanded to meet the needs of the audience. Exhibit 19-2 lists four types of reports, the sections that 
are typically included, and the general order of presentation. Each of these formats can be modified to 
meet the needs of the audience. 


>Exhibit 19-2 Research Report Sections and Their Order of Inclusion 


Short Report 


Report Modules Memo or Letter Short Technical Management 


Long Report 
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Technical 


Prefatory Information 1 1 
Letter of transmittal 
Title page 


Authorization statement 


SSS SS 


Executive summary 


ee NE a A aN 


Table of contents 


Snes Ns LoS 


Methodology v v 


Sampling design 
Research design 
Data collection 
Data analysis 
Limitations v v 


Conclusions 2 4 3 
Summary and conclusions v v v 
Recommendations v v v 


Bibliography 


be ee pa ae yay als) 
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Prefatory Items 


Prefatory materials do not have direct bearing on the research itself. Instead, they assist the reader in 
using the research report. 


Letter of Transmittal 


When the relationship between the researcher and the client is formal, a letter of transmittal should 
be included. This is appropriate when a report is for a specific client (e.g., a company officer) and 
when it is generated for an outside organization. The letter should refer to the authorization for the 
project and any specific instructions or limitations placed on the study. It should also state the pur- 
pose and the scope of the study. For many internal projects, it is not necessary to include a letter of 
transmittal. 


Title Page 


The title page should include four items: the title of the report, the date, and for whom and by whom 
it was prepared. The title should be brief but include the following three elements: (1) the variables 
included in the study, (2) the type of relationship among the variables, and (3) the population to which 
the results may be applied.' Redundancies such as “A Report of” and “A Discussion of” add length 
to the title but little else. Single-word titles are also of little value. Here are three acceptable ways to 
word report titles: 


Descriptive study: The Five-Year Demand Outlook for Consumer Packaged Goods in the 
United States 


Correlation study: The Relationship between Relative National Inflation Rates and Household 
Purchases of Brand X in International Markets 


Causal study: The Effect of Various Motivation Methods on Retail Sales Associates’ 
Attitudes and Performance 


Authorization Letter 


When the report is sent to a public organization, it is common to include a letter of authorization show- 
ing the authority for undertaking the research. This is especially true for reports to federal and state 
governments and nonprofit organizations. The letter not only shows who sponsored the research but 
also delineates the original request. 


Executive Summary 


An executive summary can serve two purposes. It may be a report in miniature (sometimes called a 
topline report), covering all the aspects in the body of the report but in abbreviated form. Or it may be 
a concise summary of the major findings and conclusions, including recommendations. Two pages are 
generally sufficient for executive summaries. Write this section after the rest of the report is finished. 
It should not include new information but may require graphics to present a particular conclusion. 
Expect the summary to contain a high density of significant terms since it is repeating the highlights 
of the report. 


Table of Contents 


As a rough guide, any report of several sections that totals more than 6 to 10 pages should have a table 
of contents. If there are many tables, charts, or other exhibits, they should also be listed after the table 
of contents in a separate table of illustrations. 
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Forrester Research: Finding the Dramatic Story Line 


Forrester Research is a firm that does issue-driven research in 
numerous industries and sells many of its reports by subscrip- 
tion. Forrester takes a modular approach to report writing, 
whether it’s a “brief” drafted in a few hours or a report that might 
take as many as 30 hours. Each report has three main sections. 
The “Market Overview” section describes the data collected from 
interviews, surveys, and secondary searches. It starts by reveal- 
ing simpler problems and moves on to more complex ones. It 
doesn’t rehash information the audience knows but provides 
only those data that are new. The “Analysis” section interprets 
the findings. And the “What It Means” section speculates on the 
implications of the findings and the analysis. 

In one such study, the Market Overview would relate the 
finding that “40 percent of the time auto dealers have the wrong 
cars.” In the Analysis section, the report would relate that with all 
the data car manufacturers have about what cars—and features 
within cars—are selling, a dealer with access to this information 
should be able to improve his or her inventory mix. Senior analyst 
Mark Bunger relates that the What It Means section is speculative. 


“We develop the W-I-M chain—if ‘a’ was found, then isn’t ‘b’ 
likely? Or if ‘b,’ then ‘c’; and if ‘c,’ then ‘d.’ A lot of deduction and 
conjecture based on solid knowledge and experience within the 
industry ends up in the last section of the report. So if 40 percent 
of the time dealers have the wrong cars, and the manufacturers 
have the information dealers need, then dealers could improve 
the inventory to reduce that rate to say 20 percent. And for those 
prospects who still can’t find the car they want? They might be 
likely to custom build to order to achieve satisfaction.” This specu- 
lative section is the smallest section of the report. 

And when it comes to writing reports, Forrester research- 
ers take the time to find the right words to relay their informa- 
tion. The title takes on special significance, as subscribers often 
choose the reports they access based on the title. “We'll get 
more people reading a report if we title it something intriguing like 
‘Will ad skipping kill television?’ than if we call it something drier 
like ‘The implications of technology on viewer control activities in 


television ad exposure. 


www.forrester.com 
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Introduction 


The introduction prepares the reader for the report by describing the parts of the project: the problem 
statement, research objectives, and background material.’ In most projects, the introduction can be 
taken from the research proposal with minor editing. 


Problem Statement 


The problem statement contains the need for the research project. The problem is usually represented 
by a management question. It is followed by a more detailed set of objectives. 


Research Objectives 


The research objectives address the purpose of the project. These objectives may be research questions 
and associated investigative questions. In correlational or causal studies, the hypothesis statements are 
included. As we discussed in Chapter 2, hypotheses are declarative statements describing the relationship 
between two or more variables. They state clearly the variables of concern, the relationships among them, 
and the target group being studied. Operational definitions of critical variables should be included. 


Background 


Background material may be of two types. It may be the preliminary results of exploration from an 
experience survey, focus group, or another source. Alternatively, it could be secondary data from the 
literature review. A traditional organizational scheme is to think of the concentric circles of a target. 
Starting with the outside ring, the writer works toward the center. The bull’s eye contains the material 
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directly related to the problem. Resources and means for securing this information are presented in 
Chapter 5 and on your text website. 

Previous research, theory, or situations that led to the management question are also discussed in 
this section. The literature should be organized, integrated, and presented in a way that connects it logi- 
cally to the problem. The background includes definitions, qualifications, and assumptions. It gives the 
reader the information needed to understand the remainder of the research report. 

Background material may be placed before the problem statement or after the research objectives. If 
it is composed primarily of literature review and related research, it should follow the objectives. If it 
contains information pertinent to the management problem or the situation that led to the study, it can 
be placed before the problem statement (where it is found in many applied studies). 


Methodology 


In short reports and management reports, the methodology should not have a separate section; it should 
be mentioned in the introduction, and details should be placed in an appendix. However, for a technical 
report, the methodology is an important section, containing at least five parts. 


Sampling Design 


The researcher explicitly defines the target population being studied and the sampling methods used. 
For example, was this a probability or nonprobability sample? If probability, was it simple random or 
complex random? How were the elements selected? How was the size determined? How much confi- 
dence do we have, and how much error was allowed? 

Explanations of the sampling methods, uniqueness of the chosen parameters, or other points that 
need explanation should be covered with brevity. Calculations should be placed in an appendix instead 
of in the body of the report. 


Research Design 


The coverage of the design must be adapted to the purpose. In an experimental study, the materials, 
tests, equipment, control conditions, and other devices should be described. In descriptive or ex post 
facto designs, it may be sufficient to cover the rationale for using one design instead of competing 
alternatives. Even with a sophisticated design, the strengths and weaknesses should be identified and 
the instrumentation and materials discussed. Copies of materials are placed in an appendix. 


Data Collection 


This part of the report describes the specifics of gathering the data. Its contents depend on the selected 
design. Survey work generally uses a team with field and central supervision. How many people were 
involved? What was their training? How were they managed? When were the data collected? How 
much time did it take? What were the conditions in the field? How were irregularities handled? In an 
experiment, we would want to know about participant assignment to groups, the use of standardized 
procedures and protocols, the administration of tests or observational forms, manipulation of the vari- 
ables, and so forth. 

Typically, you would include a discussion on the relevance of secondary data that guided these 
decisions. Again, detailed materials such as field instructions should be included in an appendix. 


Data Analysis 


This section summarizes the methods used to analyze the data and describes data handling, preliminary 
analysis, statistical tests, computer programs, and other technical information. The rationale for the 
choice of analysis approaches should be clear. A brief commentary on assumptions and appropriate- 
ness of use should be presented. 
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Limitations 


This topic is often handled with ambivalence. Some people wish to ignore the matter, feeling that men- 
tioning limitations detracts from the impact of the study. This attitude is unprofessional and possibly 
unethical. Others seem to adopt a masochistic approach of detailing everything. The section should be 
a thoughtful presentation of significant methodology or implementation problems. An evenhanded ap- 
proach is one of the hallmarks of an honest and competent investigator. All research studies have their 
limitations, and the sincere investigator recognizes that readers need aid in judging the study’s validity. 


Findings 


This is generally the longest section of the report. The objective is to explain the data rather than draw 
interpretations or conclusions. When quantitative data can be presented, this should be done as simply 
as possible with charts, graphics, and tables. 

The data need not include everything you have collected. The criterion for inclusion is, Is this mate- 
rial important to the reader’s understanding of the problem and the findings? However, make sure to 
show findings unfavorable to your hypotheses as well as those that support them, because this rein- 
forces the bond of trust that has developed between researcher and sponsor. 

It is useful to present findings in numbered paragraphs or to present one finding per page with the 
quantitative data supporting the findings presented in a small table or chart on the same page (see 
Exhibit 19-3). Although this arrangement adds to the bulk of the report, it is convenient for the reader. 


>Exhibit 19-3 Example of a Findings Page in Central City Bank Market Study 


Findings: 1. In this city, commercial banks are not the preferred savings medium. 
Banks are in a weak third place behind money market accounts. 


2. Customers of the Central City Bank have a somewhat more favorable 
attitude toward bank savings and less of a preference for government 
bonds. 


Question: Suppose that you have just received an extra $1,000 and have decided 
to save it. Which of the savings methods listed would be your 
preferred way to save it? 

Oj Government bonds 

Oj Savings and loan 

| Bank savings 

Oj Credit union 


Oj Stock 
Oj Other 
Total Central City Bank Other Bank 
Savings Method Replies Customers Customers 
Government bonds 24% 20% 29% 
Savings and loan 43 45 42 
Bank 13 18 8 
Credit union g) if afi 
Stock if 8 
Other 4 2 
Total 100% 100% 100% 
n=216 nes 1105) me nl 
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Conclusions 


Summary and Conclusions 


The summary is a brief statement of the essential findings. Sectional summaries may be used if there are 
many specific findings. These may be combined into an overall summary. In simple descriptive research, 
a summary may complete the report, because conclusions and recommendations may not be required. 

Findings state facts; conclusions represent inferences drawn from the findings. A writer is sometimes 
reluctant to make conclusions and leaves the task to the reader. Avoid this temptation when possible. 
As the researcher, you are the one best informed on the factors that critically influence the findings and 
conclusions. Good researchers don’t draw conclusions that go beyond the data related to the study. 

Conclusions may be presented in a tabular form for easy reading and reference. Summary findings 
may be subordinated under the related conclusion statement. These may be numbered to refer the 
reader to pages or tables in the findings sections. 


Recommendations 


Increasingly, researchers are expected to offer ideas for corrective actions. In applied research the 
recommendations will usually be for managerial action, with the researcher suggesting one or several 
alternatives that are supported by the findings. Also, researchers may recommend further research 
initiatives. In basic or pure research, recommendations are often suggestions for further study that 
broaden or test the understandings of a subject area. 


Appendices 


The appendices are the place for complex tables, statistical tests, supporting documents, copies of 
forms and questionnaires, detailed descriptions of the methodology, instructions to field workers, and 
other evidence important for later support. The reader who wishes to learn about the technical aspects 
of the study and to look at statistical breakdowns will want a complete appendix. 


Bibliography 


The use of secondary data requires a bibliography. Long reports, particularly technical ones, require a 
bibliography. A bibliography documents the sources used by the writer. Although bibliographies may 
contain work used as background or for further study, it is preferable to include only sources used for 
preparing the report. 


> Writing the Report 


Students often give inadequate attention to reporting their findings and conclusions. This is unfortunate. 
A well-presented study will often impress the reader more than a study with greater scientific quality but 
with a weaker presentation. Judging a report as competently written is often the key first step to a manag- 
er’s decision to use the findings in decision making and also to consider implementation of the research- 
er’s recommendations. Report-writing skills are especially valuable to the junior executive or researcher 
who aspires to rise in an organization. A well-written study frequently enhances career prospects. 


Prewriting Concerns 


Before writing, one should ask again, What is the purpose of this report? Responding to this question 
is one way to crystallize the problem. 

The second prewriting question is, Who will read the report? Thought should be given to the needs, 
temperament, and biases of the audience. You should not distort facts to meet these needs and biases but 
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should consider them while developing the presentation. Knowing who will read the report may suggest 
its appropriate length. Generally, the higher the report goes in an organization, the shorter it should be. 

Another consideration is technical background—the gap in subject knowledge between the reader and 
the writer. The greater the gap, the more difficult it is to convey the full findings meaningfully and concisely. 

The third prewriting question is, What are the circumstances and limitations under which I am 
writing? Is the nature of the subject highly technical? Do you need statistics? Charts? What is the 
importance of the topic? A crucial subject justifies more effort than a minor one. What should be the 
scope of the report? How much time is available? Deadlines often impose limitations on the report. 

Finally, How will the report be used? Try to visualize the reader using the report. How can the 
information be made more convenient? How much effort must be given to getting the attention and 
interest of the reader? Will the report be read by more than one person? If so, how many copies should 
be made? What will be the distribution of the report? 


The Outline 


Once the researcher has made the first analysis of the data, drawn tentative conclusions, and completed 
statistical significance tests, it is time to develop an outline. A useful system employs the following 
organizational structure: 


I. Major Topic Heading 
A. Major subtopic heading 
1. Subtopic 
a. Minor subtopic 
(1) Further detail 
(a) Even further detail 


Software for developing outlines and visually connecting ideas simplifies this once-onerous task. Two 
styles of outlining are widely used—the topic outline and the sentence outline. In the topic outline, 
a key word or two are used. The assumption is that the writer knows its significance and will later 
remember the nature of the argument represented by that word or phrase or, alternatively, the outliner 
knows that a point should be made but is not yet sure how to make it. 

The sentence outline expresses the essential thoughts associated with the specific topic. This ap- 
proach leaves less development work for later writing, other than elaboration and explanation to im- 
prove readability. It has the obvious advantages of pushing the writer to make decisions on what to 
include and how to say it. It is probably the best outlining style for the inexperienced researcher be- 
cause it divides the writing job into its two major components—what to say and how to say it. 

Here is an example of the type of detail found with each of these outlining formats: 


Topic Outline Sentence Outline 


|. Demand |. Demand for refrigerators 
A. How measured A. Measured in terms of factory shipments as reported to the U.S. 
1. Voluntary error Department of Commerce. 
2. Shipping error 1. Error is introduced into year-to-year comparisons because 
a. Monthly variance reporting is voluntary. 


2. Asecond factor is variations from month to month because of 
shipping and invoicing patterns. 
a. Variations up to 30 percent this year depending on 
whether shipments were measured by actual shipment 
data or invoice date. 


The Bibliography 


Style manuals provide guidelines on form, section and alphabetical arrangement, and annotation. 
Proper citation, style, and formats are unique to the purpose of the report. The instructor, program, 
institution, or client often specifies style requirements. The uniqueness of varying requirements makes 
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detailed examples in this chapter impractical, although the endnotes and references in this book provide 
an example. As cited in Appendix A on the research proposal, we recommend the Publication Manual 
of the American Psychological Association; Kate L. Turabian, A Manual for Writers of Term Papers, 
Theses, and Dissertations; and Joseph Gibaldi, MLA Handbook for Writers of Research Papers. 

Bibliographic retrieval software allows researchers to locate and save references from online 
services and translate them into database records. Entries can be further searched, sorted, indexed, 
and formatted into bibliographies of any style. Many retrieval programs are network-compatible and 
connect to popular word processors. EasyBib (www.easybib.com) and BibMe (www.bibme.org) are 
two examples. 


Writing the Draft 


Once the outline is complete, decisions can be made on the placement of graphics, tables, and charts. 
Each should be matched to a particular section in the outline. Although graphics might be added later or 
tables changed into charts, it is helpful to make a first approximation of the graphics before beginning 
to write. Choices for reporting statistics will be reviewed later in this chapter. 

Each writer uses different mechanisms for getting 
thoughts into written form. Some will write in long- 
hand, relying on someone else to transcribe their prose 
into word-processed format. Others are happiest in 
front of a word processor, able to add, delete, and move 
sections at will. Whichever works for you is the best 
approach to use. 

Computer software packages check for spelling 
errors and provide a thesaurus for looking up alterna- 
tive ways of expressing a thought. The Internet offers 
the Oxford English Dictionary, believed to be the 
greatest dictionary in any language. Common word 
confusion (there for their, to for too, or effect for 
affect) will not be found by standard spelling check- 
ers. Advanced programs will scrutinize your report 
for grammar, punctuation, capitalization, doubled 
words, transposed letters, homonyms, style problems, and readability level. The style checker will 
reveal misused words and indicate awkward phrasing. Exhibit 19-4 shows sample output from a 
commercial package used on one of this text’s vignettes. The program shown writes comments to a 
text file, prepares a backup copy of the original, and generates a statistics report. The statistics sum- 
marize the program’s evaluation of readability, grade level, and sentence structure. Comparisons to 
“reference” documents, or documents that you submit for comparison, may be made. The software 
cannot guarantee an error-free report but will greatly reduce your time in proofreading and enhance 
the style of the completed product.4 


Readability 


Sensitive writers consider the reading ability of their audience to achieve high readership. You can 
obtain high readership more easily if the topic interests the readers and is in their field of expertise. 
In addition, you can show the usefulness of the report by pointing out how it will help the readers. 
Finally, you can write at a level that is appropriate to the audience’s reading abilities. To test writing 
for difficulty level, use a standard readability index. The Flesch Reading Ease Score gives a score 
between 0 and 100. The lower the score, the harder the material is to read. The Flesch-Kincaid Grade 
Level and Gunning’s Fog Index both provide a score that corresponds with the grade level needed 
to easily read and understand the document. Although it is possible to calculate these indexes by 
hand, some software packages will do it automatically. The most sophisticated packages allow you 
to specify the preferred reading level. Words that are above that level are highlighted to allow you to 
choose an alternative. 
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>Exhibit 19-4 Grammar and Style Proofreader Results 


Statistics Comparisons 
Statistics for: Chapter Vignette Problems marked/detected: 8/8 Readability Comparison Chart 
Readability Statistics Flesch Reading Ease Score 
Flesch Reading Ease: 66 Flesch-Kincaid Grade Level: 8 Chapter Vignette es 66 
Gunning’s Fog Index: 11 Gettysburg Address «64 
—_ Hemingway short story es 8G 
Paragraph Statistics Life insurance policy a 45 
Number of paragraphs: 25 Average length: 2.2 sentences aa 
Flesch-Kincaid Grade Level 
Sentence Statistics Chapter Vignette es 8 
Number of sentences: 55 Passive voice: 4 Gettysburg Address a | | 
Average length: 13.8 words Short (< 12 words) : 39 Hemingway short story EES an 
End with “?” : 2 Long (> 28 words): 7 Life insurance policy 
End with “!” : 0 Sentence Statistics Comparison Chart 
Word Statistics ise Sen per Paragraph - 
apter Vignette a 2. 
Number of words: 759 Average length: 4.58 letters Gettysburg Address ee 
Hemingway short story 2.8 
Life insurance policy EE 3.2 


Document Summary for: Chapter Vignette Problems detected: 8 


A 7, : Average Words per Sentence 
Readability Statistics Interpretation 


Chapter Vignette BS 13.8 
Grade level: Preferred level for most readers. Gettysburg Address es 26.8 
8 (Flesch-Kincaid) Hemingway short story [Em 13.5 
Reading ease score: This represents 6 to 10 years Life insurance policy EEE 23.9 
66 (Flesch) of schooling. Word Statistics Comparison Chart 
Passive voice: Writing may be difficult to read or 
331 ambiguous for this writing style. Oa Word A5 
Average sentence length: Most readers could easily understand Gettysburg Address ee 42 
13.8 words sentences of this length. Hemingway short story = 4.0 
Average word length: Vocabulary used in this document Life insurance policy es 47 
1.50 syl. is understandable for most readers. 


Average paragraph length: Most readers could easily follow 
2.2 sentences paragraphs of this length. 


Advocates of readability measurement do not claim that all written material should be at the sim- 
plest level possible. They argue only that the level should be appropriate for the audience. They point 
out that comic books score about 6 on the Gunning scale (that is, a person with a sixth-grade education 
should be able to read that material). Time usually scores about 10, while The Atlantic is reported to 
have a score of 11 or 12. Material that scores much above 12 becomes difficult for the public to read 
comfortably. Such measures obviously give only a rough idea of the true readability of a report. Good 
writing calls for a variety of other skills to enhance reading comprehension. 


Comprehensibility 


Good writing varies with the writing objective. Research writing is designed to convey information 
of a precise nature. Avoid ambiguity, multiple meanings, and allusions. Take care to choose the right 
words—words that convey thoughts accurately, clearly, and efficiently. When concepts and constructs 
are used, they must be defined, either operationally or descriptively. 

Words and sentences should be carefully organized and edited. Misplaced modifiers run rampant 
in carelessly written reports. Subordinate ideas mixed with major ideas make the report confusing to 
readers, forcing them to sort out what is important and what is secondary when this should have been 
done for them. 
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Finally, there is the matter of pace. Pace is defined as: 


The rate at which the printed page presents information to the reader... . The proper pace in technical writing is one 
that enables the reader to keep his mind working just a fraction of a second behind his eye as he reads along. It logi- 
cally would be slow when the information is complex or difficult to understand; fast when the information is straightfor- 
ward and familiar. If the reader’s mind lags behind his eye, the pace is too rapid; if his mind wanders ahead of his eye 
(or wants to) the pace is too slow.® 


If the text is overcrowded with concepts, there is too much information per sentence. By contrast, 
sparse writing has too few significant ideas per sentence. Writers use a variety of methods to adjust the 
pace of their writing: 


Use ample white space and wide margins to create a positive psychological effect on the reader. 
Break large units of text into smaller units with headings to show organization of the topics. 
Relieve difficult text with visual aids when possible. 


Emphasize important material and deemphasize secondary material through sentence construc- 
tion and judicious use of italicizing, underlining, capitalization, and parentheses. 


Choose words carefully, opting for the known and short rather than the unknown and long. 
Graduate students, in particular, seem to revel in using jargon, pompous constructions, and long 
or arcane words. Naturally, there are times when technical terms are appropriate. Scientists com- 
municate efficiently with jargon, but the audiences for most applied research are not scientifi- 
cally trained and need more help than many writers supply. 


Repeat and summarize critical and difficult ideas so that readers have time to absorb them. 


Make strategic use of service words. These are words that “do not represent objects or ideas, 
but show relationship. Transitional words, such as the conjunctions, are service words. So 
are phrases such as ‘on the other hand,’ ‘in summary,’ and ‘in 
contrast.’ ’”¢ 


Tone 


Review the writing to ensure the tone is appropriate. The reader can, and 
should, be referred to, but researchers should avoid referring to them- 
selves. One author notes that the “application of the ‘you’ attitude. . . 
makes the message sound like it is written to the reader, not sent by the 
author. A message prepared for the reader conveys sincerity, personaliza- 
tion, warmth, and involvement on the part of the author.”” To accomplish 
this, remove negative phrasing and rewrite the thought positively. Do not 
change your recommendations or your findings to make them positive. 
Instead, review the phrasing. Which of the following sounds better? 


End users do not want the Information Systems Department 
telling them what software to buy. 


End users want more autonomy over their computer software 
choices. 


The messages convey the same information, but the positive tone of 
the second message does not put readers from the Information Sys- 
tems Department on the defensive. 


Final Proof 


Integrated office suite software like Microsoft Office has 
made it infinitely easier, and more challenging, to craftthe Tt is helpful to put the draft away for a day before doing the final edit- 


research report. Research sponsors expect articulate and 
well-organized reports and the software can deliver. How- 
ever, they also expect perfection. Only with careful editing 


ing. Go to the beach, ride a bicycle in the park, or see a movie—do 
anything that is unrelated to the research project. Then return to the 


and rewriting, coupled with skilled use of the software, is report and read it with a critical eye. Does the writing flow smoothly? 
the researcher capable of meeting that expectation. Are there transitions where they are needed? Is the organization 
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apparent to the reader? Do the findings and conclusions adequately meet the problem statement and the 
research objectives? Are the tables and graphics displaying the proper information in an easy-to-read 
format? After assuring yourself that the draft is complete, write the executive summary. 


Presentation Considerations 


The final consideration in the report-writing process is production. Reports can be typed; printed on 
an ink-jet, laser, color, or other printer; or sent out for typesetting. Most student and small research 
reports are typed or produced on a computer printer. The presentation of the report conveys to the 
readers the professional approach used throughout the project. Care should be taken to use compatible 
fonts throughout the entire report. The printer should produce consistent, easy-to-read letters on quality 
paper. When reports are photocopied for more than one reader, make sure the copies are clean and have 
no black streaks or gray areas. 

Overcrowding of text creates an appearance problem. Readers need the visual relief provided by 
ample white space. We define “ample” as | inch of white space at the top, bottom, and right-hand mar- 
gins. On the left side, the margin should be at least 14 inches to provide room for binding or punched 
holes. Even greater margins will often improve report appearance and help to highlight key points or 
sections. Overcrowding also occurs when the report contains page after page of large blocks of unbro- 
ken text. This produces an unpleasant psychological effect on readers because of its formidable appear- 
ance. Overcrowded text, however, may be avoided in the following ways: 


¢ Use shorter paragraphs. As a rough guide, any paragraph longer than half a page is suspect. 
Remember that each paragraph should represent a distinct thought. 


e Indent parts of text that represent listings, long quotations, or examples. 


e Use headings and subheadings to divide the report and its major sections into homogeneous 
topical parts. 


¢ Use vertical listings of points (such as this list). 


Inadequate labeling creates another physical problem. Each graph or table should contain enough infor- 
mation to be self-explanatory. Text headings and subheadings also help with labeling. They function as 
signs for the audience, describing the organization of the report and indicating the progress of discussion. 
They also help readers to skim the material and to return easily to particular sections of the report. 


> Presentation of Statistics® 


The presentation of statistics in research reports is a special challenge for writers. Four basic ways to 
present such data are in (1) a text paragraph, (2) semitabular form, (3) tables, or (4) graphics. 


Text Presentation 


This is probably the most common method of presentation when there are only a few statistics. The 
writer can direct the reader’s attention to certain numbers or comparisons and emphasize specific 
points. The drawback is that the statistics are submerged in the text, requiring the reader to scan the 
entire paragraph to extract the meaning. The following material has a few simple comparisons but 
becomes more complicated when text and statistics are combined: 


Walmart regained its number-1 rank in the Forbes 500 due to its strong sales performance 
(11% increase; $351.1 billion). Although Walmart surpassed number-2-ranked Exxon Mobil 
in sales, Walmart’s profitability ($11.2 billion) was far below the oil giant ($39.5 billion). 
Some credit several challenging public relations problems with the lower-than-expected level. 
Number-6-ranked General Electric also outperformed Walmart in profits with $20.8 billion. 
GE’s robust sales growth (27.4%) is an indication that it will likely challenge both Walmart 
and ExxonMobil in the future. 
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This section 
continues 
on p. 529 
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MindWriter Written Report 


{) Awritten report is the culmination of the MindWriter 
project, which has illustrated the research pro- 
MindWriter\ Ges throughout the book. The contract for the 
CompleteCare project requires a report about the size of a stu- 
dent term project. Although repetitive portions have been omit- 
ted to conserve space, it should give the reader some idea of 
how an applied project of this size is summarized. Descriptive 
statistics and simple graphics are used to analyze and present 


most of the data. References to chapters where specific details 
may be reviewed are shown in the marginal comments. 

The presentation of findings follows the content specifica- 
tions of Exhibit 19-2 for short reports. It falls between a memo/ 
letter and a short technical report. The objective was to make it 
available quickly for feedback to the CompleteCare team. It was 
therefore set up as a PDF e-mail attachment. 


The fax cover sheet acts 
as a temporary transmittal 
letter until the plain paper 
copies are sent. 


To: Myra Wines From: Jason Henry 

Company: MindWriter Corp. Company: Henry and Associates 
It provides all necessary Location: Austin, TX Bldg 5 Location: Palm Beach, FL 
identification and contact Telephone: = 512.555.1234 Telephone: 407.555.4321 
information. The writer's Fax: 512.555.1250 Fax: 407.555.4357 


and recipient's relationship 
makes using first names 
appropriate. 


January 5, 2014 


Authorization for the study. 
Scope of findings (month). 
Specific instructions for 
process issues. 


Dear Myra, 


Request for follow-up by 
the client to reduce the 
study’s limitations. 


Progress update 
and feedback on 
improvements. 


Best regards, 


Jason 


Total number of pages including this one: 11 


This fax contains the CompleteCare December report requested by Mr. Malraison. 
You may expect the plain paper copies tomorrow morning for distribution. 


We hope that the Call Center will complete the nonrespondent surveys so that we can 
discover the extent to which these results represent all CompleteCare customers. 


This month’s findings show improvements in the areas we discussed last week by 
telephone. The response rate is also up. You will be delighted to know that our prelim- 
inary analysis shows improvements in the courier’s ratings. 


Title contains reference 
to a known survey and 
program. Descriptions of 
variables, relationships, 
and population are 
unnecessary. 


The recipient of the report, 
corporation, and date 
appear next. 


The report’s preparer, 
location, and telephone 
number facilitate contact 
for additional information. 


The information level 
identifies this as a 
restricted circulation 
document for in-house 
use only. 
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CompleteCare Customer 
Survey Results for December 


Prepared for Myra Wines 
MindWriter Corporation 
January 2014 


Henry and Associates 
200 ShellPoint Tower 
Palm Beach, Florida 33480 


407.555.4321 


e-mail: info@henry&assoc.com 
www.henry&assoc.com 


MindWriter CONFIDENTIAL 
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Title repeated. 


Section headings are 
used. 


Introduction contains 
period of coverage for 
report, management 
question, and secondary 
research objective. 


An overview of the report’s 
contents allows readers to 
turn to specific section of 
interest. 


The executive summary 
provides a synopsis of 
essential findings. It is the 
report in miniature —six 
paragraphs. 


Both positive and negative 
results are capsulized. 


Criteria for indexes are 
provided as reminders. 
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MindWriter CompleteCare December Results 


Introduction 


This report is based on the December data collected from the MindWriter Complete- 
Care Survey. The survey asks customers about their satisfaction with the Complete- 
Care repair and service system. Its secondary purpose is to identify monthly 
improvement targets for management. 


The findings are organized into the following sections: (1) an executive summary, 
(2) the methods used, (3) the Service Improvement Grid, (4) detailed findings for 
each question, and (5) patterns in the open-ended questions. 


Executive Summary 


The highest degrees of satisfaction with CompleteCare were found in the categories 
of “delivery speed” and “pickup speed.” Average scores on these items were between 
4.2 and 4.4 on a 5-point scale. “Speed of repair,” “condition on arrival,” and “overall 
impression of CompleteCare’s effectiveness” also scored relatively well. They were 
above the met all expectations level (see appropriate charts). 


Several questions were below the met all expectations level. From the lowest, “Call 
Center's responsiveness,” to “Call Center's technical competence,” and “courier ser- 
vice’s arrangements,” the average scores ranged from 2.0 to 3.9. In general, ratings 
have improved since November with the exception of “condition on arrival.” 


The three items generating the most negative comments are (1) problems with the 
courier's arrangements, (2) long telephone waits, and (3) transfer among many people 
at the Call Center. These same comments carry over for the last two months. 


CompleteCare’s criteria for Dissatisfied Customers consist of negative comments in 
the Comments/Suggestions section or ratings of less than three (3.0) on questions 
one through eight. Forty-three percent of the sample met these criteria, down from 
56 percent last month. By counting only customers’ comments (positive/negative 
or +/-), the percentage of Dissatisfied Customers would be 32 percent. 


The ratio of negative to positive comments was 1.7 to 1, an improvement over 
November's ratio (2.3 to 1). 


The methodology, 
reported in brief, reminds 
the readers of the data 
collection method, 
nature and format 

of the questionnaire, 
scales used, and target 
measurement issues. 


The sample, a self- 
selecting nonprobability 
sample, and the response 
rate are shown. With 
respondents’ data from 
the postcards and the 
Call Center’s files on 
nonrespondents, a future 
study on nonresponse 
bias is planned. 


This section begins the 
Findings section. Findings 
consist of the action 
planning grid and detailed 
results sections. The 
headings were specified 
by the client. 


The method for creating 
the planning grid and the 
grid’s contents are 
highlighted. 
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When the expectation-based satisfaction scores are adjusted for perceived impor- 
tance, “Call Center responsiveness,” “Call Center technical competence,” and 
“courier arrangements” are identified as action items. “Repair speed” and “problem 
resolution” maintained high importance scores and are also rated above average. 


Methodology 


The data collection instrument is an online survey. Participation is solicited via 
email among users of the CompleteCare program. 


The survey consists of 12 satisfaction questions measured on 5-point scales. The 
questions record the degree to which the components of the CompleteCare process 
(arrangements for receiving the customer's computer through return of the repaired 
product) meet customers’ expectations. A final categorical question asks whether 
customers will use CompleteCare again. Space for suggestions is provided. 


Sample 


The sample consisted of 175 customers who provided impressions of CompleteCare’s 
effectiveness. For the four-week period, the response rate was 35 percent with no 
incentive given. Nothing is yet known about the differences between respondents and 
nonrespondents. 


Service Improvement Grid 


The grid on page three compares the degree to which expectations were met along 

with the derived importance of those expectations. The average scores for both axes 

determine the dividing lines for the four quadrants. The quadrants are labeled to iden- 
tify actionable items and to highlight those that bear watching for improvement or dete- 
rioration. 


The Concentrate Efforts quadrant is the area where customers are marginally sat- 

isfied with service but consider service issues important. Question 1a, “Call Center's 
responsiveness,” Question 1b, “Call Center's technical competence,” and Question 
2a, “courier arrangements,” are found here. “Technical competence” was similarly 
rated last month. Its perceived importance was rated higher in previous months. 
“Courier arrangements” has increased in perceived importance over previous reports. 
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The statistical technique 
for producing the grid is 
correlation. A modification 
of scatterplots was 

used to create a plot 

with reference lines (see 
Chapter 18). 


The contents of each 
quadrant are described. 
Comparisons and 
connections to the next 
section are previewed. 
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High 
Concentrate efforts Good work 
15 
Courier arrangements @ 

70 Center’s technical e e Repair speed 

; competence Problem 
‘ ; resolution 
8 Center s @ Courier 
rt responsiveness pickup 
5 ~65 
roe 
£ 


Condition 
® on arrival 


Courier 
e delivery speed 


Overkill 


Low priority 


Low 1.5 2.0 2.5 3.0 3.5 4.0 4.5 High 
CompleteCare Satisfaction 


Note: Satisfaction scores are in the range of 1.0 to 5.0 and importance is in the range of 0 to 1.0. 


In the Good Work quadrant, CompleteCare has, on average, met all expectations 

with the “repair speed” and “courier pickup” questions. Their mean scores are greater 
than 4.0 and considered important by respondents. “Problem resolution” has 
improved but remains a borderline concern. 


There are no items in the Low Priority quadrant. 


Overkill, the last quadrant, contains two questions. Question 5, “condition on arrival,” 
has improved its ratings over last month but has dropped slightly on the importance 
scale because the average of importance scores (horizontal line) moved upward. 
Question 2c, “courier delivery speed,” has a high satisfaction rating, but respondents 
considered this item to have lower importance than most issues in CompleteCare. 


Detailed findings show 
the results of individual 
questions. This section 
announces the two-part 
content and presents, 
briefly and in a direct 
style, the most pertinent 
outcomes. 


This graphic gives the 
reader a three-month view 
of all the questions at a 
glance. Vertical bars are 
the simplest and easiest 
to read for the amount of 
space allocated. Horizontal 
grid lines guide the eye 
from the bar tops to the 
closest value on the mean 
score axis. 


Charts similar to these 
may be produced by 
the same spreadsheet 
that handles data entry. 
Charting programs offer 
other options and will 
import the data from 
spreadsheets. 


>chapter 19 Presenting Insights and Findings: Written Reports 523 


>closeupcont’d 


Detailed Findings 


The figures that follow provide (1) a comparison of the mean scores for each of the 
questions for the last three months and (2) individual question results. The latter con- 
tains frequencies for the scale values, percentages for each category, mean scores, 
standard deviations, and valid cases for each question. (See Appendix for question 


wording and placement.) 


The three-month comparison (October, November, December) shows results for all 
scaled questions. December data bars (in dark blue) reveal improvements on all 
average scores (vertical axis) except Question 5, “condition on arrival.” Most aspects 
of the service/repair process have shown improvement over the three-month period. 


Monthly Comparison 


ja 1b 2a 2b 2c 38 4 5 6 7 8a 8b 


Dec. Means: 


1.98 2.89 3.89 4.19 4.44 4.17 4.10 4.14 4.14 4.54 4.02 4.09 


October 


[Mi November [J December 
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The first individual item Question 1a. Call Center’s Responsiveness. This question has the lowest 

is reported with mean mean score of the survey. Using a top-box method of reporting (combining the top 
two categories), 11 percent of the respondents felt that the Call Center met or 
exceeded their expectations for service responsiveness. This has improved only mar- 
ginally since November and has significant implications for program targets. Based on 


scores, percentages, and 
recommendations for 


Hnplevemete our visit and recent results, we recommend that you begin immediately the contin- 
gency programs we discussed: additional training for Call Center operators and imple- 
mentation of the proposed staffing plan. 

Call Center’s Responsiveness 
100 

This chart conveys 90 

the message of low 80 

responsiveness rather 

well but does not have a 

a label for the vertical 60 

axis. It is easy to confuse 50 

percentages with the 40 

number of respondents 30 

(which it is supposed to 

represent). 20 

10 
0 

Similar reporting formats Met few Metsome Met most Met all Exceeded 

are skipped. Expectations 


Mean score: 1.98 Standard deviation: 1.09 Valid cases: 159 


Question 6 shows 

the respondents’ 

overall impression of 
CompleteCare. It would 
be an ideal dependent 
variable for a regression 
study in which questions 1 
through 5 were the 
independent variables 
(see Chapter 18 and the 
Online Learning Center 
supplement Multivariate 
Analysis, An Overview). 


Question 8a is another 
question for more 
detailed research. It 
allows the researcher to 
connect the variables 
that describe the service/ 
repair experience with 
repurchase intentions. 


>chapter 19 Presenting Insights and Findings: Written Reports 525 


>closeupcont’d 


Question 6. Overall Impression of CompleteCare’s Effectiveness. 
CompleteCare has increased the number of truly satisfied respondents with 46 per- 
cent (versus 43 percent in November) in the exceeded expectations category. The 
top-box score has increased to 75 percent of respondents (against 70 percent in 


November). 


100 


Overall Impression of CompleteCare’s Effectiveness 


Met few Metsome Met most Met all Exceeded 
Expectations 


Mean score: 4.14. Standard deviation: 0.98 Valid cases: 169 


Question 8a. Likelihood of Repurchasing MindWriter Based on 
Service/Repair Experience. Respondents’ average scores (4.02) for this likeli- 
hood scale are the highest this month since measurement began. Improvement of the 
courier service’s arrangements with customers and the resolution of the problem that 
prompted service appear to be the best predictors of repurchase at this time. 
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Using regression made 


it possible to identify two Likelihood of Repurchasing MindWriter 
key influences for this Based on Service/Repair Experience 


question. 100 
90 
Question 8b (not 80 
shown) is similar, asking 70 
about the relation of 60 
product performance to 
repurchase intention. 50 
40 
30 
20 
10 
0 
Very Unlikely Neither likely Likely Very likely 
unlikely nor unlikely 


Mean score: 4.02 Standard deviation: 1.10 Valid cases: 165 


Patterns in the Open-Ended Questions 


The questionnaire has one The following categories were found when the comments and suggestions were 


open-ended question that analyzed. The ratio of negative to positive comments was 1.7 to 1. Pickup problems 
encourages respondents continue to be “courier only” problems and coordination between MindWriter’s 
to make comments or telephone support and the courier. Customers complain of holding on the phone for 
suggestions. long periods and being transferred between support people. Problems with service 


are split between large problems that have not been fixed and small, nuisance 
problems that customers are prepared to live with. Positive comments commend 
turnaround and service and also praise specific technical operators. 


Content analysis is used to 
distill the responses (see 
Chapter 15). 


>chapter 19 Presenting Insights and Findings: Written Reports 527 


>closeupcont’d 


Although content 
analysis produces 
more than frequency 
counts of recurring 
themes, it is a labor- 
intensive process. The 
project’s restrictive 
budget and the needs 
of the audience made 
this section of the 
report adequate for its 
purpose. 


Negative Comments Count 


Shipping 19 


Pickup problems (15) 


Delivery problems (2) 


Box damage (1) 


The courier charged customer (1) 
Call Center 19 
Too long on hold (9) 


Transferred call too frequently/confusion (8) 
Untrained/hard to understand (2) 
Service 13 


Problem continues (5) 


Small things not fixed/damaged (6) 


Took too long (2-7 weeks) (2) 
Product 6 
Multiple repairs needed (3) 


Paint wears off (2) 


General dislike of product (1) 


Positive Comments 


General positive comment about the process 13 
Quick response 12 
Great service 7 


Helpful phone personnel 6 


Other Comments 


MindWriter shouldn’t need to be repaired 


Provide more information on what was done 


Offer extended warranty 


+ = f BK 


Won’t use MindWriter Call Center again 
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This report’s appendix 
contains a copy of the Appendix: Questionnaire 
questionnaire. 


MindWriter personal computers offer you ease of use and maintenance. When you need 
service, we want you to rely on CompleteCare, wherever you may be. That’s why we’re 
asking you to take a moment to tell us how well we’ve served you. 


Please answer the first set of questions using the following scale: MindWriter 
Met Few Met Some Met Most Met All Exceeded 
Expectations Expectations Expectations Expectations Expectations 
1 2 3 4 5 

1 2 3 4 5 


1. Telephone assistance with your problem: 
a. Responsiveness 
b. Technical competence 
2. The courier service’s effectiveness: 
a. Arrangements 
b. Pickup speed 


c. Delivery speed 


OOO0O000 


3. Speed of the overall repair process 
4. Resolution of the problem that prompted service/repair CQ 


5. Condition of your MindWriter on arrival 


CFO MORO OMIO 
L2OMPORIORIORMIO 
L2O MO RIOR OMO 
DOMOMIOMOmMO 


O 
6. Overall impression of CompleteCare’s effectiveness O 
How likely would you be to... 


Very Somewhat Neither Somewhat Very 
Unlikely Unlikely —_— Unlikely Likely Likely 


nor Likely 
7. Use CompleteCare on another occasion O O O O O 
8. Repurchase another MindWriter based on: 
a. Service/repair experience O O O O O 
b. Product Performance O O O O O 


Please share any additional comments or suggestions 


How may we contact you to follow up on any problems you have experienced? 


Last Name First Name Email 
City State Zipcode Phone 


Service Code 


Thank you for your participation. 


SUBMIT 
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Semitabular Presentation 


When there are just a few figures, they may be taken from the text and listed. Lists of quantitative com- 
parisons are much easier to read and understand than embedded statistics. The following is an example 
of a semitabular presentation is shown: 


Walmart slipped to No. 2 in the 2011 Fortune 500 after holding onto the top spot for two years in a 
row. The retail giant was forced to aggressively cut prices to reverse its declining same-store sales 
in the United States. 


¢ Walmart is the second largest business in the Fortune 500 with revenues up by 6 percent but 
profits down by 4.2 percent. 


e Oil and energy giant Exxon Mobil is the most profitable business in the Fortune 500 with rev- 
enues up 27.7 percent and profits up 34.8 percent. 

¢ While Exxon Mobil benefited from rising oil prices, it also positioned itself well to capitalize on 
the latest trend in domestic energy production: fracking. 


¢ Despite rising oil prices, Chevron’s year ended with a sour note with a fourth quarter loss; how- 
ever it maintained its number 3 ranking with annual revenues and profits soaring, 25.1 percent 
and 41.4 percent, respectively. 


Tabular Presentation 


Tables are generally superior to text for presenting statistics, although they should be accompanied by 
comments directing the reader’s attention to important figures. Tables facilitate quantitative compari- 
sons and provide a concise, efficient way to present numerical data. 


Walmart slipped to No. 2 in the 2011 Fortune 500 after holding onto the top spot for two years in a 
row. The retail giant was forced to aggressively cut prices to reverse its declining same-store sales 
in the United States. 


How Walmart 2011 Revenues Revenue Growth Profits Profit Growth 
Compares Rank ($, millions) over Prior Year ($, millions) over Prior Year 


3 245,621.0 25.1% 26,895.0 


Exxon Mobil 


Chevron 


Source: “2011 Forbes 500,” CNN Money, accessed June 26, 2012 (http://money.cnn.com/magazines/fortune/ 
fortune500/201 2/snapshots/387.html; http://money.cnn.com/magazines/fortune/fortune500/201 2/snapshots/ 
2255.html; http://money.cnn.com/magazines/fortune/fortune500/201 2/snapshots/385.html). 


Tables are either general or summary in nature. General tables tend to be large, complex, and de- 
tailed. They serve as the repository for the statistical findings of the study and are usually in the ap- 
pendix of a research report. 

Summary tables contain only a few key pieces of data closely related to a specific finding. To make 
them inviting to the reader (who often skips them), the table designer should omit unimportant details and 
collapse multiple classifications into composite measures that may be substituted for the original data. 

Any table should contain enough information for the reader to understand its contents. The title 
should explain the subject of the table, how the data are classified, the time period, or other related mat- 
ters. A subtitle is sometimes included under the title to explain something about the table; most often 
this is a statement of the measurement units in which the data are expressed. The contents of the col- 
umns should be clearly identified by the column heads, and the contents of the stub should do the same 
for the rows. The body of the table contains the data, while the footnotes contain any needed explana- 
tions. Footnotes should be identified by letters or symbols such as asterisks, rather than by numbers, to 
avoid confusion with data values. Finally, there should be a source note if the data do not come from 
your original research. Exhibit 19-5 illustrates the various parts of a table. 
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>Exhibit 19-5 Sample Tabular Finding 


Percent Internet Use among U.S. Adults* < Title 


Use 
Internet Use Home High Speed __ Internet for Make a 
at Least Receive Internetor Connection Info about Phone Call 
Occasionally Email Email Use at Home Hobby Online 


Percent Distribution 


Education 
Less than High School 40 27 73 12 385 11 
High School Graduate 68 63 85 86 62 ilo 
Some College 86 84 91 92 48 22 
College Graduate 93 92 96 94 42 29 


Ss 
T 
U Age Body 
B 
18-29 91 83 89 91 48 33 
30-49 86 80 89 90 51 20 
50-64 73 69 91 90 51 12 
Income 
Less than $30,000 60 58 79 83 51 12 
$30,00-49,999 80 79 90 88 48 23 
$50,000-74,999 88 85 97 94 oN 20 
$75,000+ 97 64 96 96 44 30 
*Refers to adults 18 or older, with access within the 30 days prior to their participation in the tracking study. __] Footnote 
Source: PEW Research Center’s Internet & American Life Project, Tracking Study, July 25—-August 26, 2011 "| Source 


(Tables: AACTIV31, AACTIV12, |_User, Home3NW, Modem3b, Emlocc, IntUse.) note 
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Graphics 


Compared with tables, graphs show less information and often only approximate values. However, 
they are more often read and remembered than tables. Their great advantage is that they convey quan- 
titative values and comparisons more readily than tables. With personal computer charting programs, 
you can easily turn a set of numbers into a chart or graph. 

There are many different graphic forms. Exhibit 19-6 shows the most common ones and how they 
should be used. Statistical explanation charts such as boxplots, stem-and-leaf displays, and histograms 
were discussed in Chapter 16. Line graphs; area, pie, and bar charts; and pictographs and 3-D graphics 
receive additional attention here. 


Line Graphs 


Line graphs are used chiefly for time series and frequency distribution. There are several guidelines 
for designing a line graph: 


¢ Put the time units or the independent variable on the horizontal axis. 


e When showing more than one line, use different line types (solid, dashed, dotted, dash-dot) 
to enable the reader to easily distinguish among them. 


e Try not to put more than four lines on one chart. 


e Use a solid line for the primary data. 


It is important to be aware of perceptual problems with line diagrams. The first is the use of a zero 
baseline. Since the length of the bar or distance above the baseline indicates the statistic, it is important 
that graphs give accurate visual impressions of values. A good way to achieve this is to include a zero 
baseline on the scale on which the curves are plotted. To set the base at some other value is to introduce 
a visual bias. This can be seen by comparing the visual impressions in parts A and B of Exhibit 19-7. 
Both are accurate plots of cable television systems in the United States from 1985 through 2012. In 
part A, however, using the baseline of zero places the curve well up on the chart and gives a better 
perception of the relation between the absolute size of cable systems and the changes on a five-year 
interval. The graph in part B, with a baseline at 35 million, can easily give the impression that the 
growth was at a more rapid rate. When space or other reasons dictate using shortened scales, the zero 
base point should still be used but with a break in the scale as shown in part C of Exhibit 19-7. This will 
warn the reader that the scale has been reduced. 

The balance of size between vertical and horizontal scales also affects the reader’s impression of the 
data. There is no single solution to this problem, but the results can be seen by comparing parts B and 
C in Exhibit 19-7. In part C, the horizontal scale is twice that in part B. This changes the slope of the 
curve, creating a different perception of growth rate. 

A third distortion with line diagrams occurs when relative and absolute changes among two or more 
sets of data are shown. In most charts, we use arithmetic scales, where each space unit has identical value. 
This shows the absolute differences between variables, as in part A of Exhibit 19-8, which presents the 
light-and heavy-truck sales in the United States from 1985 to 2011. This is an arithmetically correct way to 
present these data; but if we are interested in rates of growth, the visual impressions from a semi-logarith- 
mic scale are more accurate. A semi-logarithmic scale uses a logarithm along one axis (usually the vertical 
or Y axis) and an arithmetic scale along the other axis (usually the horizontal or X axis). The Y axis shows 
quantity, and the X axis shows time. Arithmetic data are converted to natural logs by spreadsheet or statis- 
tical software and plotted. Semi-logarithmic graphics preserve percentage relationships across the scale. 

A comparison of the line diagrams in parts A and B of Exhibit 19-8 shows how much difference a 
semi-logarithmic scale makes. Each is valuable, and each can be misleading. In part A, notice that sales 
of both light and heavy trucks have grown since 1985 but heavy-truck sales are only a small segment of 
U.S. sales of trucks and have a much flatter growth curve. One can even estimate what this proportion 
is. Part B gives insight into growth rates that are not clear from the arithmetic scale. It shows that while 
light trucks had a major growth spurt between 1985-2011, a spurt not shared by heavy trucks, since 
then their growth patterns have been more consistent with each other. From the calculated growth rate, 
in two of the last four five-year periods examined, the growth in heavy-truck sales actually exceeded 
the growth in light-truck sales, even while light-truck sales far exceeded heavy-truck sales. 
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>Exhibit 19-6 Guide to Charts for Written Reports 


For Components of a Whole or Frequency 


9 a\= 


Pie Shows 
relationship of parts 
to the whole. Wedges 
are row values of 
data. 


Exploded Pie 
Draws attention to 
critical component 
within the whole. 


For Relationships or Comparisons 


Simple Bar Places 
categories on the 

Y axis and amounts 
or percentages on 
the X axis. 


all 


Simple Column 

Places categories on 
the X axis and amounts 
or percentages on the 
Y axis. 


Pictograph 
Represents values 
as pictures; either 
bar or column. 


iniHi 


Stacked Bar 
Shows amounts 
of component 
variables; either 
bars or columns. 


Bar Compares 
different entities on 
the same variable or 
component of a 


Bullet Bar Compares 
different entities on 
the same variable or 
component of a 


Column Compares 
different entities on 
the same variable or 
component of a 


Deviations (Bar or 
Column) Positions 
categories on X axis 
and values on Y axis. 


Mirror Image Bar 
Positions categories 
on Y axis and values 
on X axis as mirror 


Area (surface) Like 
line chart, compares 
changing values but 
emphasizes relative 


variable. variable. variable. Deviations distinguish images for different value of each series. 
positive from negative entities. 
values. 
— 
\ 4 
f \ 
/1\ 
| . 
& Jal I 
& e id 


Bubble Used to 
introduce third 
variable (dots of 
different sizes). Axes 
could be sales, profits; 
bubbles are assets. 


Line Compares 
values over time to 
show changes in 
trends. 


Filled Line Similar 
to line chart, but uses 
fill to highlight series. 


Boxplots Displays 
distribution(s) and 
compares 
characteristics of 
shape (Chapter 17). 


Step Compares 
discrete points on 
the value axis with 
vertical lines showing 
difference between 
points. Not for 
showing a trend. 


Column with Line 
Item of most interest is 
presented in bars and 
compared to items 
represented by lines; 
categories on X axis 
and values on Y axis. 


Stacked Pie Same 
as pie but displays 
two or more data 
series. 


Multiple Pie Uses 
same data as stacked 
pie but plots separate 
pies for each column 
of data without 
stacking. 


Scatter Shows if 
relationship between 
variables follows a 
pattern. May be used 
with one variable at 
different times. 


Spider (and Radar) 
Radiating lines are 
categories; values 
are distances from 


center (shows multiple 


variables—e.g., 
performance, ratings, 
progress). 


Side-by-Side 
Stacked Bar 
Compares 
components of two 
or more items of 
interest. 
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>Exhibit 19-7 Cable Subscribers, 1985-2012 
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>Exhibit 19-8 U.S. Truck Sales 1985-2011 (in thousands) 


A. Arithmetic scale 


B. Semi-logarithmic scale 


10000 10000 
9000 ee aa ee 
8000 
7000 1000 
6000 ee 
5000 100 
4000 
3000 
2000 i 
1000 
0 0 
1985 1990 1995 2000 2005 2010 2011 1985 1990 1995 2000 2005 2010 2011 
—s Light trucks —e- Heavy trucks —s- Light trucks -e- Heavy trucks 
Sales 1985 1990 1995 2000 2005 2010 2011 
Heavy trucks 295 277.5 388.4 461.9 496.5 217.6 306.1 
Light trucks 4446.7 4559.5 6081.4 8491.6 9228.4 5826 6540.7 
Percent Growth in Sales 
1985-1990 1990-1995 1995-2000 2000-2005 2005-2010 2010-2011 
Heavy trucks —5.9 40.0 18.9 7.5 —56.2 40.7 
Light trucks 2.5 33.4 39.6 8.7 —36.9 12.3 


Source: Data were extracted from Michigan Senate, accessed June 25, 


Economics/RetailAutoSales.PDF). 


2012 (http://www.senate.michigan.gov/sfa/ 
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>Exhibit 19-9 Examples of Area Charts: A Stratum Chart and Two Pies 


Notice that the two pie charts seem to indicate a decrease in the “under 25” category relative to the stratum chart. 
The “under 25” category did in fact decrease (from 40 to 33 percent) but not as dramatically as the stratum to pie 
comparison would suggest. Also note that the sample size changed from 100 to 180 units between 1975 and 2005. 
It is important not to use a pie chart alone in a time series, to avoid giving erroneous impressions. 


A Stratum Chart Pie Charts 

200 
180 
160 
140 
120 
100 
80 


1975 


1975 1985 1995 2005 2015 


2005 


|| 55 and over 25-54 


Area (Stratum or Surface) Charts 


An area chart is also used for a time series. Consisting of a line that has been divided into component 
parts, it is best used to show changes in patterns over time. The same rules apply to stratum charts as 
to line charts (see Exhibit 19-9). 


Pie Charts 


Pie charts are another form of area chart. They are often used with business data. However, they can 
easily mislead the reader or be improperly prepared. Research shows that readers’ perceptions of the 
percentages represented by the pie slices are consistently inaccurate.’ Consider the following sugges- 
tions when designing pie charts: 


e Show 100 percent of the subject being graphed. 


e Always label the slices with “call-outs” and with the percentage or amount that is represented. 
This allows you to dispense with a legend. 


° Put the largest slice at twelve o’clock and move clockwise in descending order. 
¢ Use light colors for large slices, darker colors for smaller slices. 


e Ina pie chart of black and white slices, a single red one will command the most attention and be 
memorable. Use it to communicate your most important message.'° 


¢ Do not show evolution over time with pie charts as the only medium. Since pie charts always 
represent 100 percent, growth of the overall whole will not be recognized. If you must use a 
series of pie charts, complement them with an area chart. 


As shown in Exhibit 19-9, pie charts portray frequency data in interesting ways. In addition, they 
can be stacked to show relationships between two sets of data. 
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Bar Charts 


Bar charts can be very effective if properly constructed. Use the horizontal axis to represent time 
and the vertical axis to represent units or growth-related variables. Vertical bars are generally used for 
time series and for quantitative classifications. Horizontal bars are less often used. If neither variable is 
time-related, either format can be used. A computer charting program (e.g., Excel, the newest version 
of SPSS) easily generates charts. If you are preparing a bar chart by hand, leave space between the bars 
equal to at least half the width of the bar. An exception to this is the specialized chart—the histogram— 
in which continuous data are grouped into intervals for a frequency distribution (see Chapter 17). A 
second exception is the multiple-variable chart, in which more than one bar is located at a particular 
time segment. In this case, the space between the groups of bars is at least half the width of the group. 
Bar charts come in a variety of patterns. In Chapter 17, Exhibit 17-3 shows a standard vertical bar 
graph. Variations are illustrated in Exhibit 19-6. 


Pictographs and Geographs 


These graphics are used in popular magazines and newspapers because they are eye-catching and 
imaginative. USA Today and a host of imitators are often guilty of taking this to the extreme, creating 
graphs that are incomprehensible. A pictograph uses pictorial symbols (an oil drum for barrels of oil, 
a stick figure for numbers of employees, or a pine tree for amount of wood). The symbols represent 
data volume and are used instead of a bar in a bar-type chart. It is proper to stack same-size images to 
express more of a quantity and to show fractions of an image to show less. But altering the scale of the 
symbol produces problems. Since the pictures represent actual objects, doubling the size will increase 
the area of the symbol by four (and the volume by more). This misleads the reader into believing the 
increase is larger than it really is. The exception is a graphic that is easily substituted for a bar, such as 
the pencils in Exhibit 19-6. 

Geographic charts use a portion of the world’s map, in pictorial form, to show variations in re- 
gional data. They can be used for product sales, distribution status, media consumption, promotional 
response rates, per capita rates of consumption, demographics, or any of a number of other geographi- 
cally specific variables. 

Stacked data sets produce variables of interest that can be aligned on a common geographic referent. 
The resulting pictorial display allows the user to “drill” through the layers and visualize the relation- 
ships. With better Windows-based software and government agencies providing geo-codes and refer- 
ence points, geographic spatial displays are becoming a more common form of graphic. 


3-D Graphics 


With current charting techniques, virtually all charts can now be made three-dimensional. Although 
a 3-D graphic adds interest, it can also obscure data. Care must be used in selecting 3-D chart candi- 
dates (see Exhibit 19-10). Don’t confuse pie and bar charts that have achieved dimensionality simply 
by adding depth to the graphics; this is not 3-D. A 3-D column chart allows you to compare three or 
more variables from the sample in one bar chart—-type graph. If you want to display several quarters of 
sales results for Hertz, Avis, Budget, and National, you have 3-D data. Surface charts and 3-D scatter 
charts are helpful for displaying complex data patterns if the underlying distributions are multivariate. 
Finally, be careful about converting line charts to ribbon charts, and area charts to 3-D area charts; 
these can be hard for a novice to read and your primary objective in graphical presentation is always 
data clarity. 
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>Exhibit 19-10 3-D Charts 


3-D Column 

A variation on column 
charts, they compare 
variables to each other 
or over time. Axes: 

X = categories, 

Y = series, Z = values. 
Other variations include 


3-D Ribbon This 
example is a one-wall 
plot showing columns 
of data (Series) as 
ribbons. One or more 
columns are used. 
Axes: X = categories, 
Y = series, Z = values. 


3-D Surface Line 
Handles three columns 
of data and plots XYZ 
suitable for changes in coordinates to show a 
time and multivariate response surface. 
data. Axes: X = categories, Helpful for multivariate 
Y = series, Z = values. applications. 


3-D Wireframe 
A variation of a contour 
or response surface; 


3-D area charts and 
connect-the-dots 
scatter charts. 


1 A quality presentation of research findings can have an 


inordinate effect on a reader’s or a listener’s perceptions of 
a study’s quality. Recognition of this fact should prompt a 
researcher to make a special effort to communicate skillfully 
and clearly. 


Research reports contain findings, analysis, interpretations, 
conclusions, and sometimes recommendations. They may 
follow the short, informal format typical of memoranda and 
letters, or they may be longer and more complex. Long 
reports are of either a technical or a management type. 
In the former, the problem is presented and followed by 
the findings, conclusions, and recommendations. In the 


3 The writer of research reports should be guided by four 


questions: 
e What is the purpose of this report? 
Who will read it? 


e What are the circumstances and limitations under which 
it is written? 


e How will the report be used? 


Reports should be clearly organized, physically inviting, and 
easy to read. Writers can achieve these goals if they are care- 
ful with mechanical details, writing style, and comprehensibility. 


There is a special challenge to presenting statistical data. 


management report, the conclusions and recommenda- 
tions precede the findings. The technical report is targeted 
at the technically trained reader; the management report is 
intended for the manager-client. 


While some of these data may be incorporated in the text, 
most statistics should be placed in tables, charts, or graphs. 
The choice of a table, chart, or graph depends on the spe- 
cific data and presentation purpose. 
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Terms in Review 
1 Distinguish between the following: 


a Technical report and management report. 
b Topic outline and sentence outline. 


Making Research Decisions 
2 What should you do about each of these? 
a Putting information in a research report concerning the 
study’s limitations. 
b The size and complexity of tables in a research report. 
c The physical presentation of a report. 
d Pace in your writing. 


3 What type of report would you suggest be written in each of 
the following cases? 

a The president of the company has asked for a study of 
the company’s pension plan and its comparison to the 
plans of other firms in the industry. 

b You have been asked to write up a marketing experi- 
ment, which you recently completed, for submission to 
the Journal of Marketing Research. 

c Your division manager has asked you to prepare a fore- 
cast of promotional budget needs for the division for the 
next 12 months. 

d_ The National Institutes of Health has given you a grant to 
study the relationship between advertising of prescription 
drugs and subsequent sales of those drugs. 


4 There are a number of graphic presentation forms. Which 

would you recommend to show each of the following? Why? 

a Acomparison of changes in average annual per capita in- 
come for the United States and Japan from 1995 to 2010. 

b The percentage composition of average family expen- 
diture patterns, by the major types of expenditures, for 
families whose heads are under age 35 compared with 
families whose heads are 55 or older. 

c Acomparison of the changes in charitable giving 
between December 31, 2006, and December 31, 2010. 


5 Use Exhibit 19-2 and plan the structure of your course proj- 
ect or of a research study you have read about in one of the 
Snapshots in this text. 


6 Choose any case containing data from the text website and 
prepare a findings page, similar to the one in Exhibit 19-3. 


7 Executives at Warner Brothers believe that Ellen DeGeneres 
is the heir apparent to Oprah Winfrey in daytime TV. In its 
last season, a research report revealed that The Ellen De- 
Generes Show was, for the first time, on par with The Oprah 
Winfrey Show in viewers’ minds. Implementing the findings, 
Warner Brothers announced that NBC’s 10 owned-and- 
operated stations would keep broadcasting DeGeneres’s 
show through 2014. 

The written report revealed that DeGeneres’s hour-long 
show is upbeat and inspirational—traits that appeal to day- 
time’s female audience; she is seen as relaxed and relatable; 
and she is more likable than Winfrey, according to the Q 
Scores Company. Even though the DeGeneres show lagged 
behind the Winfrey show in the treasured Nielsen ratings, on 
emotional issues DeGeneres was in a virtual tie with Winfrey. 


a What other questions should the report address? 


b How would you advise the networks to “franchise” 
DeGeneres based on your study’s results? 


From the Headlines 

8 The Index of Small Business Optimism declined 3 points in 
June 2012, falling to 91.4 and losing gains achieved earlier in 
the year, according to the NFIB Research Foundation report. 
Labor market indicators and spending plans for inventories 
and capital equipment drove about 40 percent of the decline. 
Small-business owners reluctance to expand remained histor- 
ically high due to political uncertainty. The report’s Optimism 
Index contains questions on employment, capital outlays, 
inventories, the economy in general, sales, credit, expansion, 
and earnings. What other factors affect small-business confi- 
dence and how would you integrate them into the report? 


eT 


Inquiring Minds Want to Know—WOW! 


Mastering Teacher Leadership 


NCRCC: Teeing Up and New Strategic Direction 


Ohio Lottery: Innovative Research Design Drives 
Winning 


Proofpoint: Capitalizing on a Reporter’s Love of 
Statistics 


“You will find a description of each case in the Case Index section of the textbook. Check the Case Index to determine 
whether a case provides data, the research instrument, video, or other supplementary material. Written cases are 
downloadable from the text website (www.mhhe.com/cooper1 2e). All video material and video cases are available 
from the Online Learning Center. The film reel icon indicates a video case or video material relevant to the case. 
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learningobjectives 


After reading this chapter, you should understand .. . 


How the oral research presentation differs from and is similar to traditional public speaking. 

Why historical rhetorical theory has practical influence on business presentation skills in the 21st century. 

How to plan for the research presentation. 

The frameworks and patterns of organizing a presentation. 

The uses and differences between the types of materials designed to support your points. 

How proficiency in research presentations requires designing good visuals and knowing how to use them effectively. 
The importance of delivery to getting and holding the audience's attention. 


Why practice is an essential ingredient to success and how to do it; and, what needs to be assembled and checked 
to be certain that arrangements for the occasion and venue are ready. 


6 GListeners have one chance to hear your talk and 
can’t “re-read” when they get confused. In many 
situations, they have or will hear several talks on the 
same day. Being clear is particularly important if the 


audience can’t ask questions during the talk. 9 9 
Mark D. Hill, 


professor of computer sciences and electrical and computer engineering, 


University of Wisconsin-Madison 


bringingresearchtolife ...<° 


“Based on the poll results that are on your screen, you 
have reached a strong consensus on your first priority. 
The research strongly supports that you should be 
negotiating stronger courier contracts to address the 
in-transit damage issues. Congratulations,” concluded 
Jason. 

“That wraps up our briefing, today. Sara and I are 
happy to respond to any e-mail questions any of you 
might have after reading the summary report that has 
been delivered to your e-mail. Our e-mail address is on 
screen, and it is also on the cover of the report. Myra, 
I’m handing control of the meeting back to you.” 

As Myra started to conclude the meeting, Sara was 
holding up a sign in front of Jason that read. “Turn off 
your microphone.” Jason gave a thumbs-up sign and 
clicked off his mic. 

“Thank you, Jason,” stated Myra. “The research 
has clarified some critical issues for us and you have 
helped us focus on some probable solutions. This 
concludes the meeting. Ill be following up soon with 
an e-mail that contains a link to the recorded archive 
of this presentation, allowing you to share it with your 
staff. You will also be asked to participate in a brief 
survey when you close the Web-presentation window. 
Id really appreciate your taking the three minutes it 
will take to complete the survey. Thank you all for 
attending.” 

As soon as the audience audio was disconnected, 
Myra indicated, “That went well, Jason. The use of 
the Q&A tool to obtain their pre-report ideas for action 
was a stroke of genius. When you posted the results as 
a poll and had them indicate their first priority, they 
were all over the board. It helped them understand that 


one purpose of the research and today’s meeting was to 
bring them all together.” 

“Sara gets the credit for that stroke of genius,” 
claimed Jason after removing his microphone and 
clicking on his speakerphone. “She is a_ strong 
proponent of interaction in our briefings. And she 
continually invents new ways to get people involved 
and keep them engaged.” 

“Kudos, Sara,” exclaimed Myra. “Who gets the 
credit for simplifying the monthly comparison chart?” 

“Those honors actually go to our intern, Sammye 
Grayson,” shared Sara. “I told her while it was a 
suitable graph for the written report; it was much too 
complex a visual for the presentation. She did a great 
job. P'Il pass on your praise.” 

“Well,” asked Myra, “where do we go from here?” 

“Jason and I will field any questions for the next 
week from you or your staff,” explained Sara. “Then 
we will consider this project complete—until you 
contact us again.” 

“About that,” Myra paused, “I’ve just received an 
e-mail from Jean-Claude. He wants to meet with you 
both about a new project he has in mind. He asks if he 
could pick you up at the Boca airport on Friday, about 
2:30 p.m. He says his flying office will have you back 
in time for an early dinner.” 

Sara consulted her iPhone and indicated she was 
available. Jason looked at his own calendar and smiled 
across the desk at Sara. “Tell Jean-Claude we’ll meet 
him at the airport. Any idea what this new project is 
about?” 

“Not a clue!” 
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> Introduction 


Researchers frequently present their findings orally. Exhibit 20-1 introduces the process and indicates 


its fit within the research process model. 


A research presentation has some unique characteristics that distinguish it from other kinds of public 
speaking but with which it shares similarities. A small group of people is normally involved; statistics 
often constitute an important portion of the topic; the audience members are usually managers with an 
interest in the topic, but they want to hear only the data and conclusions that will help them make criti- 
cal decisions; speaking time will often be as short as 20 minutes but may run longer than an hour; and 


>Exhibit 20-1 Oral Presentations and the Research Process 


Determine Oral Presentation Venue 


In Person Web-based 


Small Large Small Large 
Group Group Group Group 


Determine Written Report Access 


Non-Electronic 


Develop Appropriate Data Displays 


Electronic 


Compile & Practice Presentation 


Deliver Research Briefing 


Address Post-Briefing Questions 
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>Exhibit 20-2 A Model for Presentation Planning 


the presentation is normally interspersed with questions and discussion. In this chapter, we cover the 
essential ingredients for a successful presentation or briefing: how to plan, organize, support, visualize, 
deliver, practice, and arrange your presentation. 

A successful presentation requires condensing a lengthy and complex body of information. Speak- 
ing rates should not exceed 100 to 150 words per minute; thus, a 20-minute presentation limits you to 
about 2,000 to 3,000 words. If you are to communicate effectively under such conditions, you must 
plan carefully. Begin by asking two questions. First, how long should you plan to talk? Usually the 
research sponsor indicates the acceptable presentation length. Following organizational custom, an 
organization may allot a given amount of time for such briefings. If the time is severely limited, then 
the need for topical priorities is obvious. This leads to the second question: What is the purpose? Is it 
to raise concern about problems that have been uncovered? Is it to add to the knowledge of audience 
members? Is it to give them conclusions and recommendations for their decision making? This chapter 
answers these questions and leads to a plan for your success. In Exhibit 20-2 you will note that the outer 
ring of the concentric circles coincides with the organization of this chapter. The next ring represents 
the type of proofs that are required in all presentations, as described by Aristotle. Finally, the dynamics 
of the speaker, speech, occasion, and audience lead to the effect that the presenter seeks. 


> Aristotle’s Three Principles of Persuasive 
Communication 


Most readers recognize Aristotle as the authority who developed a comprehensive theory of rhetoric. 
Does a Greek philosopher, born in 384 Bc, still have influence on business presentation skills in the 
21st century? We think so. Aristotle considered rhetoric as the ability to see persuasive possibilities 
in every presentational situation and gave us a method to discover all the means of persuasion on any 
topic whatsoever. “Aristotle is important precisely because the rhetoric he taught was inventional, con- 
cerned with developing the best possible story, rather than aiming at being elegant, or ornamental, or 
passionate or beautiful or even at being a post-modernist rhetoric of giving voice to marginal people.” 
Aristotle’s advice is as relevant today as during his time and should be the basis of developing and 
improving our presentational skills. 

The basis of persuasion was defined by Aristotle with his three principles of proof: ethos, pathos, 
and Jogos. He associated communication with persuasion and identified communication as the ability 
to discover, in any given case, the available means to achieve persuasion.’ If we think of persuasive 
discourse on a continuum from a conscious attempt to modify thinking to influencing the behavior 
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and actions of the listener, then it is clear that all communication is persuasion, or at least includes a 
persuasive component. Understanding the communication process from this perspective seems more 
practical than artificially categorizing the types of presentations we give in business settings by purpose 
(informational, ceremonial, entertaining, and persuasive)—because more than one purpose is inevita- 
bly involved. 


Ethos 


Our perception of a presenter’s character affects how believable or convincing we find that person. The 
projection of credibility via personal character is called the speaker’s ethos. A strong research presenta- 
tion relies on a researcher’s ability to convince his or her audience of the following: 


1. That he or she is credible. 
2. That the findings from the research are credible. 


3. That the audience should act upon the findings, as well as conclusions and recommendations 
drawn from these findings. 


Ethos relies on how well the audience believes that the presenter is qualified to speak on the par- 
ticular subject.* To inspire confidence in the speaker, Aristotle says that three things move us to belief 
apart from any proof: good sense, goodwill, and good moral character. Revealing these personal char- 
acteristics in your delivery can play a large role in gaining credibility for your ideas. People whose 
education, experience, and previous performance qualify them to speak on a certain issue earn the spe- 
cial extrinsic ethos of authority when their reputation is known ahead of time.t However, without prior 
experience, research presenters must borrow it by linking their methodology and procedures to credible 
sources with experience. In a culture where outward appearances have virtually taken over from the 
inner appeals of character (moral and intellectual), the appeal from ethos can be both problematic and 
advantageous. 


Pathos 


Rhetoricians over the centuries consider pathos the strongest of the appeals. Pathos relies on an emo- 
tional connection between the speaker and his or her audience. It involves an appeal to an audience’s 
sense of identity, self-interest, and emotions. These appeals take advantage of common biases: we 
naturally move in the direction of what is advantageous, what serves our interests, or the interests of 
any group we are a part of. 

People hear messages based on their state of mind. If their emotional disposition is positive, they 
are more likely to be receptive to the message; if it is negative, they will be less receptive to the mes- 
sage.° The research presenter must arouse emotions exactly because they have the power to modify 
the audience’s predispositions and, thus, its judgments. Knowing the audience’s predisposition (e.g., 
resistance or skepticism or receptivity) and predetermining a desired emotional response encourages 
the presenter to build the content and delivery of a presentation to stimulate a desired emotional state. 
Open-mindedness is the most important desired pre-presentation emotional state for any audience, and 
receptivity is the most desired post-presentation emotional state. 


Logos 


Finally, with Jogos, the logical argument, we find the explicit reasons that the speaker needs to support 
a position. This translates into supporting evidence and analytical techniques that reveal and uphold the 
researchers’ findings and conclusions (described in more detail later in this chapter). 

Appeals to Jogos most often use a variation of the syllogism called an enthymeme. The syllogism is 
a formal method of deductive reasoning (described in Chapter 3). You may recognize the enthymeme 
as a truncated syllogism where one or more minor premises are left unstated. This is done because 
people do not naturally speak in syllogistic form. The presenter gives the primary premise and expects 
that the audience will supply the missing knowledge (premises) in order to reach the conclusion. In 
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>Exhibit 20-3 The Role of Aristotle’s Proofs in Persuasive Communication 


A Speaker ... 


Creates personal 
character (ethos), 
emotional (pathos) 
and logical (logos) 
proofs 


Arranges those 
proofs strategically 


Packages 


The supporting 
images and materials 
in a clear, compelling 
fashion 


Invention Organization Support/Style Delivery 


Source: Adapted from Raymie McKerrow, Bruce E. Gronbeck, Douglas Ehninger, and Alan H. Monroe, Principles 
and Types of Speech, 14th ed. (Boston, MA: Allyn & Bacon, 1999). 


a research presentation, this planning is done in the audience analysis stage when estimating the au- 
dience’s knowledge, predisposition, and attentiveness. However, most research presentations use the 
enthymeme to move from data to interpretations to conclusions. For example: “We do not have a suf- 
ficient advertising budget to improve consumers’ perceptions of our second tier brand. The brand may 
falter and hinder our ability to remain competitive. We should divert resources to remain competitive.” 

Logos is the core of most research presentations; it is normally used to describe facts and findings 
that support the speaker’s contentions about research results but it should not be the only content of the 
presentation. Since data are not likely to be manipulated by a trusted source, Jogos may sway cynical 
listeners. But, data can also be misleading, inaccurate, or unethically applied thereby eroding the good- 
will and credibility previously established by the presenter. Researchers prone to build their presenta- 
tions solely on Jogos reduce the likelihood that they will achieve their desired result—implementation 
of recommended actions inherent in the research findings. The role of Aristotle’s three proofs is sum- 
marized in Exhibit 20-3. 


> Plan 


Where do we start to prepare for the research presentation? This is the dilemma faced by beginners and 
seasoned experts. Perhaps the more pertinent question is how do we start? From a Zen perspective, if 
we start from a beginner’s mind we can see things more clearly, enthusiastically, and unburdened by 
fixed views. As Garr Reynolds, in Presentation Zen, advises, “If your state of mind is coming from 
a place of fear and risk avoidance, then you will always settle for the safe solutions—the solutions 
already applied many times before.” Some research companies use templates that can be modified 
for specific presentations. But if multiple presentations are made to the same audience of managers 
using such a template, presentations may fail to hold attention and communicate persuasively. Thus, an 
attitude of openness produces a fresh approach to planning your presentation. 

Authors with similar viewpoints ask us to consider spending most of our time in thinking, sketching, 
and scripting. We are counseled to have a story to tell before even thinking of opening PowerPoint, 
and have a 90-to-| ratio of preparation to delivery time; in other words, plan in analog. Planning 
in analog involves all the things we should do outside the digital domain. A three-step process, 
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The Culture of Reporting 


One of the few universally followed rules in presenting research 
results is to craft the message to fit the client. When the Team 
One/Lexus team ventured to Japan to share their early research 
findings, they knew they had to deliver any negatives with a polite, 
highly sensitive approach. The news was good. But to engineers 
who had crafted the Lexus SC 430 to be twice as good as the 
Jaguar XK8— more comfortable, quieter, and easier to handle— 
some of the findings would be puzzling. The team had held three 
static product clinics —-where more than 250 luxury buyers were 
assembled to compare, but not drive, the Lexus SC 430 and 
its competitors. Shortly thereafter they had conducted numerous 
focus groups of the top tier of these interested luxury buyers, 
known as acceptors. Among the early findings the team learned 


depressed, to show exhaust, and to handle more like a sports car, 
“Culturally, Japanese engineers have come to see themselves— 
and with justification—as entitled to make a car the way a car 
should be made,” shared Arian Barrow, account manager for 
Lexus at Team One Advertising. This made telling them what they 
would consider negatives somewhat difficult. For example, buy- 
ers’ expectations were that the car would zoom from zero to 60 
in under five seconds, not arrive there in eight or nine seconds. 
“So we found ourselves sharing results in a less hard-hitting way 
than we would with a different client.” How did they deliver the 
unexpected news? “People loved the car! But they would love it 
even more if it would go from zero to 60 in five seconds!” 


www.teamoneadv.com; www.lexus.com 


that buyers expected the car to growl when the accelerator was 


writing—sketching—producing requires us to plot the story like a movie script. Using paper and pen to 
sketch ideas in the analog world leads to greater creativity and clarity for the finished product.’ The 
consultant behind Al Gore’s global warming documentary, An Inconvenient Truth, suggests that the 
speaker should plan to spend as much as 90 hours to create a 1-hour presentation containing 30 slides.® 
Our general checklist, which starts the planning process with five critical W’s, accentuates the audi- 
ence’s role in the planning process. 


¢ Who makes up the audience? 

e What do they want to learn about? 

e Why is this presentation occurring and how does it connect to the larger picture? 

¢ When will the presentation take place and what are the time-of-day considerations? 


e Where will the presentation take place—including nature of the venue and travel?? 


Audience Analysis'® 


You have already noticed that most of these questions are audience-centered. So, let’s look at some ways 
we can analyze the audience more effectively. First, good speakers understand that the primary purpose 
of their presentation is to gain a desired response from their listeners. The ultimate success of their pre- 
sentation depends on the speaker’s ability to anticipate audience response. An analysis of the expected 
attendees at a presentation or audience analysis is accomplished by keeping three questions in mind: 


1. Who will I be addressing? 
2. Why should my listeners really care about the information I present? 


3. What do I want the audience to know, believe, and or do because of my presentation? 


The answers to these first two questions help develop the pathos of your presentation. Your most 
important understanding about audiences is that they are egocentric; they pay attention to messages 
that affect them directly. Their mantra is “Why should I care?” Elements of pathos can be discovered 
by collecting past impressions from prior associations; interviewing critical members of the intended 
audience; or, often less feasibly, surveying a sample of the invitees regarding agreement with a series 
of statements about issues (to determine predispositions).'' The answers to the third question help 


>chapter 20 Presenting Insights and Findings: Oral Presentations 


develop the /ogos of your presentation. Your research findings represent the core of the Jogos of your 
presentation. 

Answers to these questions come from understanding the psychology of the audience. Several psy- 
chological principles have important implications for speakers. 


e An audience member comes to the presentation venue with past knowledge of the speaker’s 
topic and will judge the presentation based on selective perception (what they know and 
believe). 


¢ As the audience listens to the speech and hears threads of information consistent with accumu- 
lated knowledge, their processed meanings will fluctuate between agreement/disagreement and 
clarity/confusion. 


¢ As the speaker addresses an audience, he or she must attempt to imaginatively construct how the 
audience will interpret the message. 


e Each audience member organizes his or her unique construct of the content presented, which is 
dependent on listener experience and openness to change." 


Demographic and dispositional audience characteristics also play an important role in assessing 
the answer to the first question. Audience composition in business now reflects a different mix than 
that of a few years ago. Speakers must adapt to the age of the group because generational similarities 
in experience and value affect receptivity. Gender can have a strong influence on audience response. 
In a multicultural and multiracial society, every audience will have a slightly different response to the 
speaker—just as religious views and cultural sensitivity need to be considered in a more narrow range 
of business situations. Finally, education, economic status, position in the organization, and group 
membership provide additional clues as to interest and attitude. The remote audience for a presentation 
(using Web services to present and connect) requires more, not less analysis, as the presenter has to 
work doubly hard to establish and maintain a connection. 

The second question considers the disposition of the audience and their needs and attitudes. Target- 
ing your presentation to fulfill appropriate needs differentiates successful presentations from those that 
fail. Knowing the needs of key audience members either through informal, advance conversations with 
the sponsor or psychological profiling can be critical to success. For example, once known, you are able 
to tailor your presentation to such need orientations as physiological, knowledge, social, or ego. And, 
you can have a good idea about the predispositions of decision makers regarding the importance they 
attach to achievement, status, career, recognition, or adherence to organizational norms. 

General features of the presentation situation and unique features of the audience are also consid- 
erations of audience analysis. What is the physical setting of the presentation? The room size, seating 
arrangement, and temperature will all affect listeners. A large audience size may require a more formal 
presentation, or affect your language choice and visual aides. Presenting to an individual or a small group 
may dictate an informal briefing rather than a formal presentation. Although we will cover staging and 
arrangements in a later section, the audience affects these issues and should be part of your analysis. In 
Exhibit 20-4, we pose seven important questions to help you understand the nature of your audience. 


Types of Learners 


In planning for your presentation, it is wise to also consider that the audience is composed of three 
types of learners; the proportionate composition of learner types will vary based on topic and age 
group. Speakers can make an emotional connection with their audiences to the extent that they rec- 
ognize the differences among visual, auditory, and kinesthetic learners. Audience members are more 
likely to act on information they have a connection with, but they cannot connect with anything that 
they have not internalized. 


e Visual learners. About 40 percent of us are visual learners, people who learn through seeing. 
This group retains information that is highly visual. To address visual learners, avoid cramming 
too much text on visual aids. Build aids that have few words and key dominant images. As many 
visual aids used in research presentations employ graphs and charts, research presentations 
already appeal to visual learners. Visual learners connect through visual imagery. 
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>Exhibit 20-4 Seven Questions to Understand Your Audience 


1. Who are they? Demographics and psychographics are a great start, but connecting with 
your audience means understanding them on a personal level. 


3. What keeps them up at Everyone has a fear, a pain point. Let your audience Know you empa- 
night? thize—and offer a solution. 


5. What do you want Make sure there’s clear action for your audience to take. 
them to do? 


7. How can you best People vary in how they prefer to receive information. This can include 
reach them? everything from the setup of the room to the availability of materials after 
the presentation. Give the audience what they want, how they want it. 


Source: Adapted from Nancy Duarte, slide:ology: The Art and Science of Creating Great Presentations. 
(Sebastopol, CA: O'Reilly Media, 2008), p. 15. 


e Auditory learners. Auditory learners represent about 20 to 30 percent of your audience. These 
people learn through listening and benefit from verbal and rhetorical techniques. Tell personal 
stories or use vivid examples to support your key messages. Research presenters can incorporate 
actual participant experiences related during the research to enrich the presentation for this type 
of listener. Auditory learners connect through stories. 


° Kinesthetic learners. These people learn by doing, moving, and touching. In short, they are 
“hands-on.” They get bored listening for long periods. Including activities in your presentation 
keeps kinesthetic listeners engaged. Pass around objects (as Steve Jobs did with the aluminum 
frame of a new laptop), conduct writing exercises, or have them participate in demonstrations." 
Research presenters can use examples of the types of exercises used with research participants, 
showcasing how research data were collected, to make a methodology come alive for this type 
of listener. Kinesthetic learners connect through activities. 


Keep Your Audience from Checking Out 


The ability for an audience to recall critical information and to avoid boredom is not, but should be, fac- 
tored into the planning of presenters. There is experimental support for the finding that recall accuracy 
varies as a function of an item’s position on a list or an argument’s sequence, referred to as the Audience 
Memory Curve" and otherwise known as the serial position effect. When asked to recall a list of items 
in any order (free recall), people recall best those items they hear at the end of the list (recency effect). 
Items at the end of the list seem to reside in short-term memory at the time of recall. Among earlier list 
items, or arguments, the first item in a list is initially distinguished as important (primacy effect) and 
may be transferred to long-term memory by the time of recall. The first few items are recalled more 
frequently than those in the middle of the presentation.'° The implication for research presenters is that 
arguments presented first or last will be highly influential to understanding and motivation to act. 
Once thought of as a “20-minute fatigue factor,’ audiences are now believed to become bored in 
10 minutes—not 11 but 10.!° According to recent research in molecular biology, the brain appears to 
be making choices according to a timing pattern influenced by genetics and environment.'’ Research 
presenters should observe the 10-minute rule by varying their content by interspersing straight talk with 
graphs, videos, demonstrations, questions, and other means that allow the brain to seek new stimuli. 


>chapter 20 Presenting Insights and Findings: Oral Presentations 


As technology has advanced the 
Internet has become a reliable 


videoconferences. As with other 
presentations, you need to be 


fails. 


Planning and the Web-Delivered Presentation 


With travel budgets reduced, managers interested in research projects often located in far-flung parts 
of the world, and the advance of Web-based technologies, web-delivered presentations are increasing. 
A Web-delivered presentation involves the use of a Web presentation platform (e.g., Live Meeting, 
WebEx, etc.), a presenter who remotely controls the delivery of the presentation visual aids to the 
audience’s computer while he or she speaks to the audience via computer or a controlled-access phone 
line, and an invited audience who participates via the Web from their office or a Web-equipped room. 
The presentation platform builds in various participation opportunities—most notably the ability of the 
audience to type in questions throughout the presentation and the ability of the presenter to respond to 
questions, or to use intermittent survey questions to poll the audience on their understanding of material 
or their consensus on a conclusion or recommendation. Some platforms offer the ability of audience 
members to ask questions of the presenter through a phone connection. Most presentations can be 
archived for later viewing, but without the opportunity to participate. The biggest problems in planning 
for a Web-delivered presentation are the lower level of audience rapport affecting the pathos of the pre- 
sentation and the longer time frame needed for planning, as is noted in Exhibit 20-5. Such a presentation 
format does, however, permit the audience to be large and offers built-in processes for pre-contact (all 
attendees must register, which is useful for audience analysis) and follow-up (useful for determining 
the effectiveness of the presentation). Web-delivered research presentations often use post-presentation 
surveys to query the audience’s understanding, solicit additional questions, and deliver copies of the 
written report. These are actions the face-to-face presentation does not often duplicate. 

We close this section with a quote from noted presenter Nancy Duarte that reminds us of the need 
to treat your audience as your first priority: “They didn’t come to your presentation to see you. They 
came to find out what you can do for them. Success means giving them a reason for taking their time, 
providing content that resonates, and ensuring that it’s clear what they are to do.”"* 


> Organize 


Presentations have an organizational structure. Without a framework, it is difficult to visualize and 
organize your content and impossible for an audience to follow and understand you. In this section, we 
review a variety of organizational strategies. 

The research literature on speech organization covers (1) optimal order of arguments, (2) one-sided 
versus two-sided arguments, and (3) the effects of organized versus disorganized messages. Studies, 
however, do not support the conclusion of one overall superior method for organizing. Instead, presen- 
tations are organized into many patterns and tailored to the speaker, content, audience, occasion and 
venue, and intended effect. The pattern you choose will depend upon the purpose of your presentation. 


medium for oral presentations and 


cautious with equipment and prepare 
for possible software glitches. Have 
a backup copy of your presentation 
on your laptop or your company’s 
server. Test your external mouse as 
well as the one that is connected to 
your computer. Be certain that your 
screensavers are disabled. And most 
important, be prepared to give your 
presentation even if the technology 
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>Exhibit 20-5 Timeline for a Web-Based Oral Research Presentation 


E-mail link 
to recorded 


session 


E-mail reminder 

with audio and 
web link 

information 


E-mail 
invitations to 
all attendees 


6 weeks 3-4 weeks 2 weeks 1 week 2 days 1 hour 
before before before before before before 


E-mail reminder 
with audio and web 
link information 


E-mail all attendees 
requesting system 
compatibility check 


Via e-mail, thank 
attendees for 
participating; 

answer any posted 

but unaddressed 
questions 


Traditional Patterns of Organization 


° Topical: there are several ideas to present, but one idea seems naturally to precede the others in 
order. In a research presentation on survey research, the presenter could organize according to 
topical areas of the questionnaire. 


° Spatial: material is organized by physical space. In a research presentation about skiing accidents 
in Colorado resorts, the presentation might be arranged by each specific ski area within Colorado. 


° Classification: puts ideas, objects, or arguments into categories. In a presentation about what 
potential customers revealed in testing prototype iPads, the order of the presentation would be 
arranged by iPad features. 


° Climax order: material is organized from the least important information to the most important. 
In a research presentation, elements are ordered to lead from the foundation findings to the key 
finding to the recommended solution. 


° Problem/solution: first part of a speech describes a problem, the middle presents research 
findings, and the second presents a solution. In research presentations, you start with the man- 
agement problem, proceed to the research problem and methodology, and conclude with the 
findings and recommendations. 


° Chronological: uses time sequence for a framework. In a research presentation on plant safety 
issues, the order would be determined by when each research activity (focus group, survey, 
experiment) occurred. 


° Past/present/future: first part section discusses the past, the second the covers the present, and 
the third predicts the future. In a research presentation using predictive modeling of inventory 
patterns, the organization starts with the data patterns of the past, then the present, then models 
the future. 
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° Cause/effect/solution: first part describes the cause of a problem, the second describes its effect, 
and the third presents a solution. In a research presentation about the effectiveness of Toyota’s 
solution for accelerator pedal malfunction, the presentation starts with the discovery of the en- 
gineering defect, describes the effect on sales, brand image, and customer loyalty; then presents 
information on what research discovered about solutions which might resonate with the auto- 
buying public. 

¢ Pros/cons/recommendation: benefits, disadvantages, and how benefits are superior are 
discussed. In a research presentation, what the researcher discovered about benefits of e-books, 
their perceived disadvantages, and what Amazon might do to enhance the advantage of the 
Kindle DX would be presented. 


e Research briefing: a research briefing is another term for the oral presentation; it starts with 
a brief statement that sets the stage for the body of the findings and explains the nature of the 
project, how it came about, and what it attempted to do. This is followed by a discussion of the 
findings that support it. Where appropriate, recommendations are stated in the third stage. 


The Motivated Sequence Organization 


One specialized pattern of organization applicable to all types of presentations is called the motivated 
sequence and is defined as “the sequence of ideas which, by following the normal processes of human 
thinking, motivates an audience to respond to the speaker’s purpose.”’!? The rationale, based on psy- 
chological principles, requires that the structure be designed to correspond to the way people habitually 
arrive at a decision despite individual differences. It consists of five steps: 


¢ Attention: draw attention to the need for change. 
° Need: call for change in existing conditions by creating dissatisfaction with them. 


e Satisfaction: satisfy the need or address the dissatisfaction with an explanation, logical demon- 
stration, or practical experience, or plan to meet objections. 


¢ Visualization: picture the benefits the proposed action will bring. 


e Action: detailed recommendation of what is needed to bring about the specified action. 


When research presentations are designed to be persuasive rather than merely informative, this or- 
ganization is powerful because it has as its goal stimulating overt action. In presentations that are more 
informative in nature, steps can be modified or deleted. 


The Narrative Organization 


The narrative pattern of presentation, or what is called narrative imaging by cognitive psychologists, 
is discussed in greater detail in the context of “stories” in the next section. The narrative pattern is 
an organizational framework that involves the use of stories as the primary vehicle for communicating 
the presenter’s message. 

We recognize that narration of stories alone may be the sole organizing mechanism for a presenta- 
tion. Later in this chapter, you will see how a single story may be woven into the presentation as one 
means of supporting a point. A legend in the speech communication field once remarked, “The only 
possible way you can undermine a conviction is to tell a story with a point. . . . [it will be effective] 
when no other type of discourse can begin to do the job—argument won’t do it, description can’t do 
it, exposition won’t do it. Narration is the only thing that can . . . tell a story. People will listen to 
stories.””° In his book, Reynolds reminds us that we are “wired” to tell and listen to stories from the 
time we were kids. Unfortunately, the use of stories in business became marginalized as synonymous 
with fiction.”! The story’s resurrection with speakers such as Steve Jobs of Apple, Howard Schultz of 
Starbucks, and John Chambers of Cisco Systems not only makes it a credible tool but opens opportuni- 
ties for new presenters to observe how the “pros” use it effectively. See the elements for constructing 
a story in Exhibit 20-6. 
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>Exhibit 20-6 Constructing a Story 


Story telling is a time-honored technique of engaging audiences. Author Cliff Atkins, in his book Beyond Bullet 
Points, suggests crafting such stories as three-act plays. Here is his format translated for a research project and 
its oral presentation. 


e Act I: Sets up the key story elements: setting, characters, conflict, desired outcome 


Affinity Bases for Athletic Alumni 


Presenter: 


Act I: Set up the story 


Indicate the problem or stimuli that led to the research in full sentence form using active 
The Setting tense, using conversational tone, limiting sentence length. 


The Protagonist The protagonist is your audience, always. 


i i i i i v7) 
The iabalance Define what is no longer as it used to be . . . why is audience here’? 


7) 
The Balance What do we want to see happen? 


How do we get there from here? 
The Solution 


e Act ll: Develops the conflict through actions and reactions of the characters in response to changing conditions 


Act Il: Develop the Action 


5-Minute Column: 15-Minute Column: 45-Minute Column: 


Exhibit 1 Quote from ... 


Exhibit 2 Quote from ... 


Exhibit 3 Quote from ... 


Exhibit 4 Quote from ... 


Exhibit 5 Quote from ... 


Exhibit 6 


Exhibit 7 Quote from ... 


O/O 
(= (i 
ome) 
2/2 
®|o 
=| 
femme) 
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>Exhibit 20-6 Constructing a Story (Continued) 


e Act Ill: ends the story, frames the resolution (climax and decision), reveals something about his/her character 


Act Ill: Frame the Resolution 


as If we don’t continually discover what motivates affinity, donations will continue to be 
The crisis F 
stagnant or decline. 
: Use online surveys to discover an understanding of affinity connectors with athletic 
The solution i ; 
alumni and other alumni groups. 
: Survey results reveal we should change everything we do—from reunion weekends to 
The climax gous : 
logo merchandise in the bookstore and online. 
The resolution Customize an affinity program for each major alumni segment. 


Research evidence on the effectiveness of the narrative organization versus the motivated sequence 
organization is informative. While the motivated sequence produces superior immediate recall, the nar- 
rative pattern of organization results in significantly more favorable attitudes toward the presentation 
and the speaker and may contribute to long-term recall.” 


The Rule of Three and the Three-Point Speech 


In the list of traditional patterns of organization, described at the beginning of this section, the last five 
examples used trios, triplets, or triads—the rule of three. This rhetorical device abounds in Western 
culture across many disciplines. It is found in religion (Three Wise Men with their gold, frankincense, 
and myrrh), movies (Sex, Lies, and Videotape), nursery rhymes (Three Little Pigs or Goldilocks and 
the Three Bears), government (Executive, Judicial, and Legislative), mottos (Fire safety: Stop, Drop, 
and Roll), and memorable speeches from Julius Caesar to Barack Obama (“We must pick ourselves 
up, dust ourselves off, and begin again the work of remaking America”).*> The favorite metaphor of a 
three-act play is a proven formula and extensively used in storytelling and screenwriting. By applying 
the rule of three as an organizing device, one is able to use any form of support, including narration. 
The result is that your presentation “gains warmth, familiarity, and understandability. With the three- 
part outline framing your ideas, your speech will be easier to follow and remember.”™ 

Is there empirical support for the rule? In 1956, Bell Labs scientist George Miller summarized 
studies that showed that individuals have difficulty retaining more than seven to nine digits in short- 
term memory.” Recent studies put the number closer to three or four; that is, working memory has a 
capacity of about four chunks in young adults and somewhat less in children and older adults.”° It is, 
therefore, no coincidence that Steve Jobs outlined a “roadmap” for his audience that is almost always 
divided into three sets: a product description with three features or a demonstration in three parts.’ 

There are many variations on the three-point speech in addition to the ones listed earlier; some of 
which you are already familiar: introduction—body—conclusion; tell them what you are going to tell 
them-tell them-then tell them what you told them (overview—body-recap); introduction-three best 
supporting points—conclusion; surprising introduction-three stories (each with points)—a memorable 
conclusion tying the stories together. Good advice about the power of repetition as a speech construc- 
tion technique is offered by Dlugan: “Take inspiration from Lewis Carroll in The Hunting of the Snark: 
‘T have said it thrice: What I tell you three times is true.’ 


> Support 


Supporting materials are the leaves on the branches of your organizational framework. After selecting 
an organizational strategy, your efforts center on compiling supporting materials to develop and vali- 
date the points you are presenting for your listener’s consideration or action. In a research presentation, 
this not only means the actual data and its interpretation but also the stories or demonstrations that 
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>Exhibit 20-7 A Checklist for Better Supporting Material 


(4 Relevant — Each piece of support should be relevant to the point it is supporting and consistent with 
the topical theme. 


(1 Appropriate — Each item of support should fit the needs and style of the receivers, meet the demands 
of that particular audience, and fit the occasion. 


(1 Believable — The material must be accurate, ethically sourced, and fairly presented. 
(1 Timely — The material must be workable within time limits. 


(1 Variety — The presentation should not rely excessively on one type of support, but should instead use 
a number of different forms of support. 


(1 Balanced — The presentation should include adequate amount of support but show a balance between 
quantity and variety, while not overburdening the case. 


(4 Speaker Specific — The material should be selected to enhance the speaker’s style of delivery as well 
as the message. 


Q Stylistic — The presentation should benefit from the power of analogies and metaphors. 


11 Simplicity — The presentation’s statistics should be conveyed in understandable terms or through 
comparisons. 


(4) Detail — Each piece of support needs to be developed to the point that audience members can 
understand and visualize how the item fits the point it is used for. 


Source: Adapted from Thomas Leech, How to Prepare, Stage, and Deliver Winning Presentations (New York: 
AMACOM, 2004), pp. 98-102; and the Speech Department at Maui Community College, http://Awww.hawaii.edu/ 
mauispeech/html/supporting_materials.html, downloaded January 27, 2010. 


corroborate the data. Supporting materials create interest, clarify the presenter’s point, provide empha- 
sis to a point, and offer proof that results in belief. Without supporting materials, an oral presentation 
is nothing more than a series of claims without evidence. See the checklist in Exhibit 20-7 for criteria 
you may use to evaluate your own materials. 

The following list presents examples of frequently used materials for supporting your presentation’ s 
arguments: 


¢ Facts are verifiable data about situations that exist or events that are known to have occurred. 
Facts often involve statistical data that can be demonstrated to be true. If true, they are not 
in dispute and thus provide powerful backing. Facts are the foundation of many research 
presentations. 


e Statistics are numerical data used in the collection, analysis, and interpretation of data, but also 
found in data collection planning, measurement, and design. Statistics are useful and expected 
for research presentations. To be a credible source of support, the listener needs to know if a 
statistic is valid and reliable, used correctly, properly interpreted, and relevant to the point. 
Statistics used sparingly reduce audience fatigue. Visuals are essential in research presentations 
to facilitate understanding of statistics. 


e Specific instance refers to a single and often critical incident selected to prove an overarching 
claim whereby specifics are translated into more general principles. These are brief rather than 
detailed stories. An instance used in the research presentation of a survey might relate the re- 
sponses of an individual respondent. 


e Example includes a single instance used to clarify a complex concept. These are often less de- 
veloped than a specific instance but have a similar advantage of helping listeners visualize the 
point. Examples may be true or hypothetical; the latter make use of a fictitious situation. In a 
research presentation, an example might relate the impact of a recommended action based on the 
responses of a single respondent or group of respondents. 
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° Testimony/Expert opinion is the perspective of recognized experts on a topic. Experts project 
credibility with your audience when used properly on the topic with which they have exper- 
tise. When used as testimonials, opinions of credible third-party endorsers allow the audience 
to absorb success stories. This is often in the form of a video clip, a quote, or participation 
of a recognized expert. These opinions represent an excellent source of support because they 
enhance ethos. 


e Analogy is the use of reasoning or is used to explain parallel cases; it is a comparison between 
two different things to highlight a point of similarity. An analogy is not offered as conclusive 
proof for an argument, but a good analogy may be useful to clarify the argument or presenter’s 
position. 

Greek philosophers Plato and Aristotle had a wider view of analogy. They saw it as a shared 
abstraction.” They believed analogous objects did not share necessarily a relation, but an idea, 
pattern, regularity, an attribute, an effect, or a function; for example,““Being obsessed with 
deficit reduction when the economy has suffered its largest setback since the Depression is like 
being obsessed with water conservation when your house is on fire—an admirable impulse, 
poorly timed.’”*° 


e Metaphor is an implicit comparison between two unlike things that actually have something 
important in common. A metaphor expresses the unfamiliar in terms of the familiar, achieving 
its effect via association, comparison, and resemblance. Metaphors‘‘carry” meaning from one 
word, image, or idea to another. Students know that when Dr. Gregory House (in the TV series 
House, M.D.) says,“T'm a night owl, Wilson’s an early bird. We’re different species,” he’s 
speaking metaphorically.*! 

Despite similarities, an analogy is not the same as a metaphor. The analogy “‘is a figure 
of language that expresses a set of like relationships among two sets of terms. In essence, the 
analogy does not claim total identification, which is the property of the metaphor. It claims a 
similarity of relationships.”*” 


Conveying Personal Experience 
through Stories 


In the previous section we introduced the power of the narrative form of organizing; now we empha- 
size individual stories as a specific type of supporting material. Stories tell the particulars of an act or 
occurrence or course of events. They are most powerful when involving personal experience. “Stories 
are who we are, and we are our stories. Good stories have interesting, clear beginnings; provocative, 
engaging content in the middle; and a clear conclusion.’”’* 

Personal experience, especially in a research presentation, links your topic to the audience and helps 
them connect with you. In a presentation, describing your experience with the study lends a real-life 
impression that cannot be achieved through facts or statistics. A personal experience helps the audience 
create a natural, emotional response. 

Examples from your world are more powerful than those you borrow. They are easier to remember 
and deliver because of their familiarity. Work, home, travel, or daily encounters provide a rich set of 
experiences and also some of the most humorous anecdotes. Professors often use self-effacing humor 
because students see themselves mirrored in our weaknesses. They are also some of the most memo- 
rable lessons for students (see Exhibit 20-8). 

Take the example of Steve Jobs’s 2005 Commencement Address at Stanford University, where he 
used the rule of three to convey three personal anecdotes about himself: “connecting the dots, love and 
loss, and a story about death.” By also finding a way to inject humor into a very serious subject, he 
created a memorable speech and one that is still frequently watched on YouTube. But more important 
than the humor was that his story was authentic; it was from his gut and his heart. Unlike his other 
performances, this one was from a manuscript, yet his stories were not memorized because they were 
real to him and consequently real to the students and families in attendance. 
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>Exhibit 20-8 Ten Steps to a Good Story 


= 


Use audience analysis to match the story to your audience and topic. 

Select language that reflects the characteristics of the audience. 

Center your story on a point and state that point clearly. 

Use the story as a mechanism to show passion and excitement about your topic. 
Learn your stories. If you forget an element, be prepared to improvise and go on. 
True facts from your life are far superior to someone else's story. 

Keep humorous stories short to capture the punch line quickly. 

Write the story out to eliminate needless diversions and unnecessary wordiness. 


Use the principle of “tweets” — 140 characters or less—to keep your sentences dramatic and punchy. 


Se one Ga & © 


ee 


Emphasize adjectives and verbs to make your stories sound more vivid. 


Source: Adapted from Advanced Public Speaking Institute, “Public Speaking Storytelling Do’s,” downloaded 
January 15, 2010 (http://www.public-speaking.org/public-speaking-storydo-article.htm). 


Demonstrations 


A good demonstration teaches. It is a variation of speech with informative intent using visual 
aids. It appeals to visual and kinesthetic learners, especially if there is an opportunity to handle the 
object. In a research presentation, the audience, through listening, watching, or participating, learns 
something new. It may be showing a new design for corporate identity or demonstrating a product 
that has resulted from your research. As the audience’s guide, you will take them through a show 
and tell process, revealing each step from start to finish. In presenting marketing and advertising 
research, a good demo informs your audience about your product, shows the benefits of ownership, 
and inspires them to take action.** Here is an example of the criteria for a great demonstration using 
an iPhone 3G: 


e Short: The EDGE versus 3G demo was less than two minutes. 

° Simple: Showed two websites loading on a smartphone. 

e Sweet: A head-to-head comparison of 3G with competitor EDGE. 

° Swift: Kept the demo moving but remained silent at key points to build drama. 


¢ Substantial: Demo resolved the problem of waiting for graphically rich sites to load.** 


> Visualize 


Proficiency in your research presentation requires the ability to create good visuals and know how to 
use them. Because visuals are so fundamental to business presentations, we use the graphics concept of 
visualization to present the material in this section. 

Eighty percent of the information humans receive comes through their eyes. For technical informa- 
tion, like research presentations, this number is probably higher.*’ Rick Altman, author of Why Most 
PowerPoint Presentations Suck and host of the PowerPoint Live User Conference, believes the real 
culprit of visually poor presentations is presenters who organize their thoughts with PowerPoint or 
other design software rather than away from the computer.*® 

Edward Tufte, the guru of displaying quantitative information says, “The PowerPoint style rou- 
tinely disrupts, dominates, and trivializes content.”*’ Another author in this chorus urges presenters 
to remember that there are three parts of a presentation: slides, notes, and handouts. Remember the 
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Preference for Pepperoni 


Among those who eat pizza at least once per month, the percent of those 
who preferred pepperoni pizza. 


Youth 14-18 Pe> Ver Vem Ve: 92% 
Adult Men @e> Bere Bee 75% 


Adult Women 53% 


>picprofile 


Because research findings are often presented orally to the research sponsor, bar charts composed of graphics that depict 
the subject of the finding may be used. In this sample slide, the researcher depicts the relative popularity of pepperoni pizza 
among different market segments. The graphic chosen should be immediately recognizable as representable of the subject. In 
this case, each pizza slice represents approximately 30% of respondents. 


difference between take-away documents (handouts) and slides. An attempt to merge the two results in 
what Reynolds calls “slideuments”—or badly detailed PowerPoint documents masquerading as slides. 
Presentations constructed this way result in the presenter supplying the verbal content of a document 
ineffectually because the audience reads faster than he or she can speak.*° 


Psychological and Physical Foundations“ 


In his book Clear and to the Point, author Stephen Kosslyn argues that audience members of any pre- 
sentation “should not have to search through a visual or conceptual haystack to find the needle you are 
talking about.” Thus the process of visualization involves developing and organizing support materials 
that help the audience share in your understanding of the data. The composition and knowledge of the 
audience, the venue, and amount of time all influence choices in visualization. 

Several psychological principles influence your audience’s understanding of your findings. The 
principle of relevance infers that only information critical to understanding should be presented. In- 
formation that is presented verbally along with visual support will be perceived as more relevant than 
that mentioned only verbally without visual support. But the principle also indicates that we do not 
want to overwhelm the audience with too much information. 

In the process of exploring your data, prior to developing a research presentation, you developed 
numerous tables, graphs, and textual summaries. Not all of these support materials, whether you use 
handouts, flip charts, or slides, can or should be used in most presentations due to time constraints. 
Any limitations in your audience’s knowledge level (principle of appropriate knowledge) or their 
inability to process large amounts of information at one time (principle of capacity limitations) re- 
duces the complexity of your support. In your attempt to share an understanding of the data, some 
support materials—for example, graphing techniques like box plots with which your audience may 
be unfamiliar—may instead create confusion or obscure the points you are trying to convey. A famil- 
iar visualization technique—a bar or column chart or table—would always convey information more 
quickly than an unfamiliar one. However, you can design even appropriate and familiar techniques in 
too complex a fashion by including unnecessary information. Your audience, after all, has only mo- 
ments to digest visually what you may have been studying for days or weeks. Exhibit 20-9 summarizes 
data graphing techniques that are appropriate for oral presentations. 
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>Exhibit 20-9 Graph Selection for Oral Presentations 


For components of a whole or frequency 


4) 8/2 


Pie: Shows 
relationship of parts 
to the whole. Wedges 
are row values 

of data. 


Exploded Pie: 
Draws attention to 
critical component 
within the whole. 


For relationship or comparisons 


Stacked Bar: 
Shows amounts 
of component 
variables; either 
bars or columns. 


Deviations 

(Bar or Column): 
Positions categories 
on X axis and values 
on Y axis. Deviations 
distinguish positive 
from negative values. 


Scatter with Trend 
Line: Shows if 
pattern exists for a 
variable; X axis and 
values on Y axis. 


Bar: 

Compares different 
entities on the same 
variable or component 
of a variable. 


Pareto Diagram: 

Item of interest is 
presented in bars 

and compared to 
aggregate represented 
by lines. 


Multiple Item Scatter: 
Multiple items are 
represented by different 
lines, with distinct 
markers for values; 
categories on X axis 
and values on Y axis. 


Simple Bar: Places 
categories on the 

Y axis and amounts 
or percentages on 
the X axis. 


Bullet Bar: 

Compares different 
entities on the same 
variable or component 
of a variable. 


aot 


Side-by-Side 
Stacked Bar: 
Compares 
components of two 
or more items of 
interest. 


a1] 


Bubble: Used to 
introduce third 
variable (dots of 
different sizes). Axes 
could be sales, profits; 
bubbles are assets. 


Simple Column: 
Places categories on 
the X axis and amounts 
or percentages on the 
Y axis. 


Column: 

Compares different 
entities on the same 
variable or component 


Pictograph: 
Represents values 
as pictures; either 
bar or column. 


Mirror Image Bar: 
Positions categories 
on Y axis and values 
on X axis as mirror 


of a variable. images for different 
entities. 
7 

Z - 

\ 
NAL VL 

V, 
Multiple Pie: Line: 


Uses same data as 
stacked pie but plots 
separate pies for each 
column of data 
without stacking. 


Compares values 
over time to show 
changes in trends. 
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According to the principle of informative changes, your audience will expect anything you speak 
about or show in your presentation to convey important information. Therefore, it is critical for your 
presentation to convey what is new or a change with a separate slide or handout, a new property 
presented on a slide or handout (e.g., flow aid change), or a new design format. Demonstrations or 
exercises relate to this principle; your audience will automatically pay more attention when you do 
something different to convey new information. 

Several additional psychological principles should also influence the visualization of your presentation. 


¢ Principle of salience. Your audience’s attention is drawn to large perceptible differences. Thus 
you should choose charting and graphing techniques that naturally showcase such differences. 
An exploding pie or a bar chart can often serve this purpose. 


¢ Principle of discriminability. Two properties must differ by a large amount to be dis- 
cerned. This means that if an important difference exists in your data but the difference is 
not large enough to be visually discernable, you must use techniques—a single highlight 
color, a supersized component, or breakouts—for the item that is important. You should 
also note whether the difference presented is statistically significant in order to establish 
importance. 


¢ Principle of perceptual organization. Your audience will automatically group items together, 
even if you do not give them such groupings. This is a mechanism they use to allow them to 
absorb and store large amounts of information. Thus, if you are trying to establish associations 
or correlations between key findings, your support materials should group themselves (in your 
organization plan, put them in proximity to each other or put them together with the same flow 
aid) and title them as a group. 


Design Principles 


In addition to selecting the right visual elements to convey your findings and conclusions, overriding 
concepts should guide the design of visual support materials.” All are achieved by what Reynolds 
describes as “careful reduction of the nonessential’**—-give the audience only what they need to under- 
stand your findings and conclusions, not everything you used to reach that level of understanding. Ideas 
reduced to their most simplistic tend to “stick” in the mind of the audience.“ 

Using the collective ideas of Reynolds, Duarte, Kosslyn, and Altman, several principles and guide- 
lines for powerful visual design emerge:* 


e Visual preparation. The presenter should conceptualize the visual support materials on paper 
before composing the digital versions. 
e A storyboard of slides, digital-camera documents, flip-chart ideas, or handouts allows the 
presenter to visually plot their argument and choose the appropriate visualization techniques. 
e Paper or sticky notes can be used to storyboard a presentation. 


¢ Flow aids. Visual techniques, such as those shown in Exhibit 20-10, convey to the audience 
where the presenter is within the overall presentation. 

e Arrow or other flow symbols or diagrams are good at denoting location and direction within 
the presentation. 

e Images and diagrams should point toward slide content or center, not to the area beyond the 
screen. 

¢ Most animations draw attention away from the message incorporated into the support 
material; use animations sparingly. 

e Animation within charts (building a chart in PowerPoint or on a flip chart) can draw audience 
attention to a desired element. 


¢ Visibility. The audience should be able to see the visual aids. 
e The bigger the room, the more likely you will need to use electronic visual aids. 
e Even small rooms need large visual aids to be seen. 
e The greater the distance between the audience and the visual aid, the larger the text and 
visual size you will need. 
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>Exhibit 20-10 Flow Aids for Structuring Presentations 


Part A: Venn Diagram Part B: Cycle Diagram 


e 


Part C: Arrow Diagram 


e Ascreen should be at least as wide as one-sixth of the distance between the screen and the 
farthest viewer from the screen. 
¢ Individual copies of support materials can replace the need for supersizing your visual aids. 


e Whitespace. Leave empty, uncluttered space surrounding important key visuals and text. 
e The audience should be able to achieve a visual focus. 
e Fewer but more compelling visual elements on each slide help achieve focus. 
¢ Bulleted keywords or phrases rather than sentences on a slide help achieve focus. 


e Picture supremacy. According to research findings, graphics and photographic images are 
more memorable than text and oral presentation.** In one study, 10 percent of information 
presented orally was remembered 72 hours after presentation, with an increase to 65 percent if a 
picture was inserted. 

e Add pictures and reduce text to improve impact. 

e Allow pictures to dominate research presentations that are less statistical. 

e When using photographs, select situations that happen in real life; realism is appreciated by 
the audience more than staged images. 


¢ Photographic framing. Create a focal point for all visuals. 

e By dividing a viewfinder by the rule of thirds, photographers compose their shots with real 
or imaginary cross-hairs that divide the field of view into thirds, vertically and horizontally. 
The cross-hairs form nine cells with four intersecting crossing points. These crossing points, 
also called power points, may be used to line up the image, creating a balanced, arresting 
visual, as well as a photograph. 

e The focal point of the image is off center—leaving whitespace around the visual. 

e Visual elements seem to present a better and more artistic sense of balance and flow. 

e The primary object is not on center, thereby avoiding predictability and redundancy and 
attracting attention. 


¢ Contrast. Use high contrast to quickly draw audience attention to the main point. 
e Busy backgrounds detract from text and images or graphs. 
° Choosing high-contrast colors (e.g., black on white, black on yellow, red on white) for text, 
data, and background or colors on charts enhances readability and denotes change. 
¢ Highlighting text in a list or numbers on a table creates more contrast than dimming out 
elements you want to recede. 
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e By varying the thickness of a line, you add to its contrast. 

¢ By adding color to an element, you add to its contrast. 

e Using too many colors for graph elements diminishes contrast; highlight what helps make 
your point. 


¢ Compatibility. The form of the message should be compatible with its content and the meaning 
of that content. 

e Organizational graphics can convey the overall presentation structure or illustrate sequences 
of steps. 

¢ Maps can be used to show location related information. 

e Data graphs can be used to reveal relative amounts. 

e Pie graphs and exploding pie graphs are used to show approximate, not absolute, proportions. 

¢ Tables convey impressions of relative amounts. 

e Line graphs can be used to convey trends over time. 

e Step graphs illustrate trends among two or more entities that vary along a noncontinuous 
scale. 

e Adjusting the size of elements (e.g., width of bars in bar chart or step graph) does not convey 
precise quantities. 

e An image or picture should be consistent with what it represents in a pictograph. 

¢ Photos or clipart can be used to define context, introduce abstract ideas, evoke emotion, or 
represent a finding or a conclusion. 


¢ Relationship. The audience should be able to see the relationships between elements and sense 
what information goes together. 

¢ The presentation organization structure should make the hierarchy obvious. 

¢ Plotting different types of data in the same display only conveys meaning if the two are 
highly related. 

¢ Plotting two types of data when they are unrelated on the same graph serves to confuse or 
clutter. 

¢ Keeping X and Y axis titles the same in comparative graphs facilitates the ability to see 
relationships. 

¢ Headers, titles, and colors can be used to group items. 

¢ Organizational diagrams that overlap, cluster elements, or radiate connecting elements can be 
used to group items. 

° Decreasing the space between bars or other graphing elements makes them easier for the 
audience to group in their mind. 


e Simplicity. Reduce clutter and use only the information and visual techniques necessary to 
convey the data, idea, or conclusion (see Exhibit 20-11). 

e Only familiar graphs, symbols, images, and jargon should be used to enhance the transition 
of information. 

e Only the part of a graph that makes your point should be used. 

¢ Numerical labels are only necessary on graphs if precision is important. 

¢ Graph legends should be eliminated if element labels are used. 

e Background grids (horizontal or vertical) should be removed if value labels are used. 

¢ Graphic or pictorial backgrounds make quick understanding of a graph impossible by 
providing too much visual stimulation. 

¢ Graphical elements (wedges in a pie, bars on a chart) should be arranged in a simple 
progression (most to least, least to most), and this arrangement should be used consistently 
from graph to graph. 

¢ Corresponding bars, from graph to graph, should be marked in the same way (same color, 
same order). 

¢ Graphics or images should always help make the presenter’s point or direct the audience’s 
attention, and not be included solely for appearance. 

e Numbered lists should be used to convey order; bulleted lists can be used when no order is 
implied. 

e Unnecessary borders (around legends, titles, axis titles) separate, rather than unify, elements. 
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>Exhibit 20-11 Simplifying Vi 


and Presentation of Data 


suals 


Department 2 


PartA YTD Expenses vs. Budget 
500 
400 
300 
200 
100 

fy) | | | 
Department 1 Department 2 Department 3 

Part B YTD Expenses vs. Budget 
Department 3 
Department 2 
Department 1 

0 50 100 150 200 250 300 350 400 
Part C YTD Variance between Budget & Expenses 

60 


Department 1 


Department 3 


—- Budget 
—— Actual 


—— Actual 
Budget 


Inappropriate technique 
because departments are 
not related. Line graphs 
are most effectively used 
to show trends over time, 
not relationships at a 
point in time. 


Appropriate technique, 
but differences do not 
stand out as quickly 
due to need for 
Department 1’s bars 

to be on the same plane 
as Department 2 and 
Department 3, which 
have smaller budgets. 


Appropriate technique 
and the differences— 
which departments are 
exceeding their budgets 
(Departments 1, 3) and 
which aren’t (Department 
2)—are quickly discerned, 
both by the chart type 
and by the color that 
separates the positive 
from the negative. 
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e Clarity. The audience should be able to perceive meaning from the location of elements. Sim- 
plicity leads to clarity and is obtained by reducing nonessential elements. Find balance between 
communicating too many research findings and being too simple. In general, however, think in 
terms of subtracting not adding. 

¢ Larger elements will be perceived to be more important than smaller ones. 

e Elements with color will be perceived as more important than those without color. 

e Fewer bullet points improve understanding. 

e Support materials should attach desired interpretation or meaning to data. 

e Mixed-media displays (e.g., bar graph overlaid with a line graph) are difficult to interpret 
during a presentation because different media do not form simple visual patterns. 


Prescriptions for Better Slides 


In summary, let us review some simple ideas for getting and holding your audience’s attention (see 
Exhibit 20-12). Go for a very low word count on your slides. The default template for PowerPoint 
promotes a two-level title and numerous subpoints. This is a slideument, which defeats the pre- 
sentation’s purpose; therefore, consider using a title only. Then heed the bullet law: if you have to 
use bullets at all, use them in moderation, and remember they should represent keywords or brief 
headlines only. Sentences should be discouraged because they take your audience’s attention away 
from the speaker while they read the sentence. Slide titles should be no more than one sentence (or 
a 140-character, memorable, Twitter-like statement). As Einstein said, “Everything should be as 
simple as possible but not simpler.” Language and visuals that are simple to parse by the audience 
are memorable. 

Use the good sense design ideas presented earlier to compose your slide. Venture capitalist, au- 
thor, and noted public speaker Guy Kawasaki also promotes the idea that presenters always need to 
simplify their message. After being the audience for numerous funding proposals by entrepreneurs, 
he developed his /0—20-30 rule: use no more than 10 slides, no more than 20 minutes (even if you 
have been allotted more time), and never use text on a slide smaller than 30 points.*’ Finding a good 
font is an art because each creates its own impression: serious or playful. But, finding its proper size is 
common sense: (1) not smaller than 30 points; (2) put your slides in the sorter option and view them 
at 66 percent—“if you can still read them, so can your audience’™*’; and (3) divide the oldest audience 
member’s age by 2 and use that font size.*? Following these suggestions, as shown in Exhibit 20-12 
will lead to simpler visuals. 


>Exhibit 20-12 Prescriptions for Better Slides 


Low word count. 


Slideuments defeat you. 


Keep it simple. 


The 10-20-30 rule. 


Font size = oldest age + 2 


Source: Based on an example from Nancy Duarte, slide:ology: The Art and Science of Creating Great 
Presentations (Sebastopol, CA: O'Reilly Media, 2008), p. 151. 


561 


562 


>part IV _ Analysis and Presentation of Data 


> Deliver 


Although the content of a presentation is a major priority, how the speaker delivers the message is 
very important. A polished presentation adds to the receptiveness of the audience, but there is always 
danger that style may overpower the message. Fortunately, the typical research audience knows why 
it is assembled, has a high level of interest, and does not need to be entertained. Even so, the speaker 
faces a real challenge in communicating effectively. In research presentations, the delivery should be 
more restrained than in those that seek action or behavioral change. Demeanor, posture, dress, and total 
appearance should be appropriate for the occasion. Speed of speech, clarity of enunciation, pauses, 
and gestures all play a part. Voice pitch, tone quality, and inflections are proper subjects for concern. 
Rapport-developing techniques are essential so that the speaker can get and hold the audience’s attention. 


Modes of Delivery 


What will be the mechanism by which you communicate your message to the audience? Will your 
presentation be memorized, read from a manuscript, or given extemporaneously? We rule out the im- 
promptu briefing because impromptu speaking does not involve preparation. Your reputation and the 
research effort should not be jeopardized by “winging it.” 

Memorization is a risky and time-consuming course to follow. Any memory slip during the pre- 
sentation can be a catastrophe, and the delivery sounds stilted and distant. Memorization virtually pre- 
cludes establishing rapport with the audience members and adapting to their reactions while you speak. 
It produces a self- or speaker-centered syndrome and is not recommended. 

Manuscript reading is also not advisable. The verbatim delivery of a presentation script sounds dull 
and lifeless because most people are not trained to read aloud, and therefore they do it badly. They become 
focused on the manuscript to the exclusion of the audience. This heads-down preoccupation with the text 
is clearly inappropriate for research presentations. If you are a trained reader with access to a teleprompter, 
this mode might work for you; most researchers do not have the skill or the equipment for such delivery. 

The extemporaneous presentation is audience-centered and made from minimal notes. This mode 
permits the speaker to be natural, conversational, and flexible. Clearly, it is the best choice for an orga- 
nizational setting. Preparation for this mode consists of writing a draft along with a complete sentence 
outline and converting the main points to notes. In this way, you can try lines of argument, experiment 
with various supporting materials, and develop memorable phraseology. Along the way, the main 
points are fixed sequentially in your mind, and supporting connections are made. 


Scripts and Notes 


Ditch the script! Manuscripts are often required for ceremonial speeches but are inappropriate for the 
majority of business presentations. They have no place in research presentations where audience mem- 
bers want to engage in information exchange, not to be read to. Imagine the audience asking itself why 
they should not be conversing with the person who wrote the script rather than the one who is reading 
it poorly to them. 

Scripts are important in the planning phase but should be shelved by the time you get to practice. 
Here are one author’s suggestions for evolving from ideation and organizing to becoming scriptless: 


e Write the script in full sentences in the notes section of PowerPoint—no more than four to five 
sentences. 


¢ Do not script every word; just rework your material removing filler words from sentences, 
leaving only key words. 


e Highlight key words and use them without regard to details to get key points into memory. 


¢ Memorize the one key idea for each slide: ask, “What is my audience’s take-home message from 
this slide?” 


¢ Practice the presentation without notes using the slide’s keywords, graph, or visual as the 
prompter.° 
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Audiences accept speaker note cards, and their presence does wonders in allaying speaker fear. 
Even if you never use them, they are there for psychological support. Many prefer to use cards for their 
briefing notes. Card contents vary widely, but here are some general guidelines for their design: 


Place preliminary remarks on the first card. 


Use the remaining cards to carry a major section of the presentation. The amount of detail 
depends on the need for precision and the speaker’s memory but should not become a mini- 
manuscript containing, for example, the details of supporting information. 


If using PowerPoint, match cards to slides. 


Include keywords and phrases, illustrations, statistics, dates, and a pronunciation guide for dif- 
ficult words. Include quotations and ideas that bear repeating. 


Along the margin, place instructions or cues, such as SLOW, FAST, EMPHASIZE, SLIDE, 
TURN CHART, and BACK TO CHART 3. 


Sequentially number your cards or notes, so you can return them quickly to order if they are 
accidentally shuffled. 


When cards are not suited to the occasion, you might consider the example in Exhibit 20-13, 
which uses the previous advice on how a slide’s keyword can serve as the prompter. In this example, 
the speaker is presenting, among other things, numerical information on the Winter Olympics in 
Vancouver, British Columbia. 


Details Make a Difference 


“Success is in the details” is our variation on German-born architect Ludwig Mies van der Rohe’s 
(1886-1969) attributed remark that “God is in the details.” In this section, we provide examples from 
often overlooked features of a presentation in a search for insights from the details. 


>Exhibit 20-13 Using Key Word Prompts as a Substitute for Notes 


T= IDI The 2012 London Olympics were the first to use HDTV freeze-frame 
technology. 


22 medals, greatest Olympian Michael Phelps increased his total Olympic medals to 22, 19 of them 
gold, swimming in three Olympics—Sidney, Beijing, and London. 


10,960 athletes 10,960 athletes competing in the 2012 London Olympics. 


160,000 tons Londoners moved tons of soil to create the London Aquatic Center. 


Sources: “Facts and figures about the Olympic Park,” accessed July 28, 2012 (http://getset.london2012. 
com/en/the-games/about-london-201 2/the-olympic-park/facts-and-figures-about-the-olympic-park). “How 
many countries are involved in the 2012 Olympics?,” accessed July 28, 2012 (http://wiki.answers.com/Q/ 
How_many_countries_are_involved_in_the_2012_Olympics). “London 2012 Olympic athletes: The full list,” The 
Guardian Datablog, accessed July 28, 2012 (http://www.guardian.co.uk/sport/datablog/201 2/jul/27/london- 
olympic-athletes-full-list#data). NBC news coverage of the Michael Phelps swimming events, August 4, 2012. 
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Clutter 


Careful language usage leads to clarity of your presentation and helps to establish one aspect of ethos, 
competence. Clutter in a speech includes repetition of fillers such as “ah,” “um,” “you know,” “like,” 
“basically,” or “exactly.” Clutter in a research presentation gives the impression of hesitancy and lack 
of competence. Caroline Kennedy was reported by Time to have botched her first interview with the 
New York press with “ums” and “you knows,” cluttering the landscape of her comments with 144 “tyou 


knows” in an 8,500-word transcript. 


Asked to justify her candidacy—after days spent with handlers advising her on how to fill Hillary Clinton’s vacant 

New York Senate seat—she began in a dull monotone: “Um, this is a fairly unique moment both in our, you know, in 
our country’s history, and, and in, in, you Know, my own life, and um, you know, we are facing, you know, unbelievable 
challenges, our economy, you know, healthcare, people are losing their jobs here in New York obviously um, arh, you 
know .. .5! 


Jargon 


Jargon, language specific to a profession or academic discipline, is considered meaningless and gen- 
erally unintelligible to people outside of that group. Using jargon is a particular danger in delivery of 
a research presentation where your audience may not be schooled in the techniques of research or of 
statistical analysis. Jargon adds clumsy language and reduces the simplicity of the message, ultimately 
confusing all but the insiders to the “code.” Jargon often needs to be defined for the audience. Unfor- 
tunately, you often learn of this only at the end of the presentation when questions are asked and you 
learn that several key points eluded the audience due to the jargon you embedded in your presentation. 
There are many types of business jargon or buzz words (see BuzzWhack.com) to avoid; here are a few 
examples:*” 


Clearer Meaning Jargon Clearer Meaning 


Swim lane Specific responsibility Move the needle Generate a reaction 


Leverage knowledge Steal someone's idea Drill down Examined more closely 
capital 
Ocular inspection Look at carefully Relanguage Reword or rewrite 


Source: Max Mallet, Brett Nelson and Chris Steiner, “The Most Annoying, Pretentious And Useless Business 
Jargon,” Forbes, January 26, 2012, accessed July 28, 2012 (http:/www.forbes.com/sites/groupthink/201 2/01/ 
26/the-most-annoying-pretentious-and-useless-business-jargon/). 


Nonverbal Communication 


Nonverbal communication is meaning conveyed through other than verbal means; it encompasses 
clothing and bodily characteristics, physical environment (physical space and time), movement and 
body position (including kinesics, posture, gesture, touch), eye gaze, and paralanguage (nonverbal cues 
of the voice). 

Nonverbal, a significant component of the speaker’s presentation, represents approximately 
50 to 60 percent of communication meaning. Some studies cite as high as 93 percent. Nonverbal 
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communication is a complex medium that can regulate the pace of your presentation, may be ambigu- 
ous, is sometimes more believable than verbal communication, and reinforces or contradicts the spoken 
message. Albert Mehrabian found a 7-38-55 rule, supposedly reflecting the percentages of how much 
communication was attributed to words, tone, and body language. Mehrabian’s research showed that 
the receiver will accept the dominant form of communication, nonverbal (38 percent + 55 percent), 
rather than the literal meaning of the words (7 percent) under conditions where a communicator is talk- 
ing about their feelings or attitudes.™ 

While a researcher making a presentation can use nonverbal communication to his or her advantage, 
it is equally important that he or she minimize distracting or contradictory nonverbal messages that 
interfere with achieving the purpose of the presentation. The complexity of this topic is such that we 
can only focus on four admonitions for the presenter. 


e Eye contact. Do you focus above people’s heads or on a wall when you present? Lack of eye 
contact is particularly bothersome to listeners and is common with inexperienced presenters. 
An important aspect of interpersonal communication is eye contact, which can also help reg- 
ulate the flow of communication with your audience. The frequency of eye contact with the 
audience helps to establish rapport and comfort, thereby increasing the speaker’s approach- 
ability. Presenters who make eye contact show concern, warmth, and authenticity. If you 
find eye contact difficult, practice using the drama coaches’ advice: scan the room slowly, 
tracing an X or Z with your eyes, but varying the pattern size to avoid looking predictable. 
Stop and look at individuals long enough to communicate with them personally before mov- 
ing on to another. 


¢ Gestures. If you do not gesture while speaking, you may be perceived as unanimated, especially 
if you keep your hands at your sides. A speaking style that is animated and lively gains audi- 
ence attention, facilitates learning, and makes your content more interesting. If you think con- 
sciously about your gestures, you are likely to gesture too late and as a result you will look stiff, 
awkward, and coached. In addition to hand gestures, facial expression (particularly smiling) is 
a powerful cue that transmits happiness, friendliness, warmth, liking, and affiliation. When you 
smile often, you will be perceived as more friendly, warm, and approachable. Luckily for the 
research presenter, gesturing to visual aids is almost mandatory and gets you started in the right 
direction. But a caution, you can overgesture in a presentation, distracting the audience with 
your body language. 


¢ Posture and body orientation. You communicate numerous messages by the way you walk 
and stand. Standing erect, but not rigid, and leaning slightly forward communicates that you 
are approachable, receptive, engaged, and friendly. Interpersonal closeness results when you 
and your audience face each other and nothing blocks the audience’s view, such as a lectern. 
Speaking with your back turned or looking too long at a slide communicates disinterest. In 
some rooms, there is too much distance between you and your audience. To offset this, mov- 
ing about the room increases interaction with your audience. Proximity enables you to make 
better eye contact and reveals your confidence. A research presenter needs to know what is 
on his or her visual aids so that the aid does not demand his or her full attention; the audience 
should have that. 


¢ Paralanguage. This facet of nonverbal communication includes such vocal elements as 
tone, pitch, rhythm, pause, timbre, loudness, and inflection. Practice varying these seven 
elements of your voice. One of the major criticisms of presenters is speaking in a mono- 
tone. The audience perceives this as boring and dull. Modulate your voice to accentuate 
key words for impact.** Vary volume, tone quality, and rate of speaking. Any of these can 
be used successfully to add interest to the message and engage audience attention. Speak- 
ers should not let their words trail off as they complete a sentence. Do you speak so softly 
that someone cannot hear you well? It is helpful to have someone in the back of the room 
signal if your voice is not carrying far enough. Do you speak too rapidly? Remind your- 
self to slow down. Make deliberate pauses before sentences. Speak words with precision 
without exaggerating. However, some people talk too slowly, and this can make the audi- 
ence restive. 
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> Practice and Arrange 


Rehearsal Is Essential 


What do super achievers and star performers have in common? Practice. Malcolm Gladwell, in his best- 
selling book, Outliers, presents the case for the “The 10,000-Hour Rule,” claiming that the key to suc- 
cess is repeated practice of a specific task for 10,000 hours. He asserts that greatness or mastery in any 
field requires massive amounts of time. With examples from the Beatles and Bill Gates, he showed how 
the Beatles performed live in Hamburg, Germany, more than 1,200 times from 1960 to 1964, accruing 
more than 10,000 hours of playing time. For Gates, it was access to a high school computer in 1968, at 
age 13, and then spending 10,000 hours programming on it.°° Whether you practice 60 hours or 6 hours, 
the time you spend rehearsing is the difference between a mastery performance and a disappointment. 
Here are some suggestions for recreating the presentational setting as you practice. The value obtained 
will pay off in preventing embarrassment, allowing you to check your ratio of material versus available 
time, uncovering holes in your supporting material, and preparing for the unexpected. 


e Practice early and often, leaving time for revision. 
e Simulate the actual setting and facilities. 


e Stand and move (reading from your computer screen does not give you realistic voice projection 
or allow you to practice movement). 


¢ Rehearse with props and visual aids. 


e Practice with an audience from your organization or people on your team who have backgrounds 
similar to the key people in the real audience. 


e Start by reading your notes or using your cue cards/slide cues to familiarize yourself with the 
flow of the presentation. 


e Stop yourself during the rehearsal to write both content and stylistic ideas as they come to mind. 
You will want to note awkward phrases that were not edited and self-conscious movements that 
were better avoided. 


e Experiment with different aspects of paralanguage (as previously discussed), gestures, or staging 
of the environment. 


e Mark your speaker note cards or the practice script as to when to pause, either for a breath or to 
stress an important point. 


e Time your presentation at least three times, or have an audience member time you. 


e Rehearse contingency plans to counter things that might go wrong. 


Now that you have had initial practice, it is time for video recording. A video of yourself speaking is an 
amazingly potent tool. Your habits—both good and bad—are captured. As you watch the video, look for: 


e Eye contact. What is the proportion of direct contact versus reading from notes? Have you used 
a simple slide cue to maintain your focus on the audience and not the slide? You should have a 
minimum of 75 percent eye contact in practice to obtain a higher goal in the presentation. 


° Body language. Are there frequent unconscious gestures like touching hair, touching your face, stand- 
ing awkwardly, pulling at your clothes? Are gestures unsynchronized with your words or unvaried? 
Do you detect body sway? If using a visual aid, demo, or a prop, are your transitions smooth? 


° Vocal characteristics. Watch for irregular breathing with long sentences, pauses in the wrong 
places, dropping or raising your voice at the end of sentences, repeated phrases (e.g., “and then 
I,” “now,” or “next”) as transitions, rapid pace of delivery, repeated fillers words, and negligible 
variation in tone or pace. 


° Energy level. Does delivering the presentation excite you? Are you enthusiastic, inspired, or 
bored? Do not underestimate the energy level necessary to generate enthusiastic listeners. One 
famous speech coach asks his clients, “On a scale of | to 10O—1 being fast asleep and 10 being 
wildly pumped up like motivational speaker Tony Robbins—where you are right now?” Most 
place themselves at 3 to 6, leaving sufficient room to raise the level.*° 
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Overcoming the Jitters 


The fear of public speaking ranks up there with the fear of 
death and/or public nudity. Whether you are a seasoned 
pro or this is your first speech, stage fright, the illogical fear 
of facing an audience, can be a paralyzing emotion. How do 
you handle those times when your mind starts going blank 
and your stomach is turning? Patricia Fripp, an award-win- 
ning keynote speaker and speech coach, provides some 
answers. She suggests that you “need to anticipate your 
speech mentally, physically, and logistically.” Mental prepa- 
ration is key and should be a six-to-one ratio: Invest three 
hours of preparation for a 30-minute speech. There is no 
substitute for rehearsal. Soend some time memorizing your 
opening and closing—three or four sentences each. Al- 
though you may speak from notes, Knowing your opening 
and closing helps your fluency, allowing you to make the 
vital connection in rapport with your audience when you are 


likely to be most nervous. The presenter’s physical appearance often reveals performance 
anxiety with perspiration, blank stares, and facial nerve tics. 


Logistically, know the room. Go there as early as pos- 
sible to get comfortable in the environment. Practice using 
the microphone and check the equipment. A quick review 
of your visual aids is also helpful. Then, during the presenta- 
tion, you can focus on your audience and not be concerned with 
the environment. 

The physical part of overcoming nervousness is varied and 
may be constrained by your setting. In a small-group setting, 
shake hands, exchange greetings, and make eye contact with 
everybody beforehand. In a larger meeting, at least connect with 
the people in the front row. Do so sincerely, and they’ll be cheer- 
ing for your success. They are not waiting for you to fail—they 
are far too worried about themselves—and they are there to lis- 
ten to you. If possible, avoid sitting while you’re waiting to speak. 
Find a position in the room where you can stand occasionally. www. fripp.com 


The rear of the room gives you access to the bathroom and 
drinking fountain. 

lf your anxiety level is still high, then you need an outlet for 
your energy. Comedians and actors find that doing light exer- 
cises in their dressing rooms or in another private area can re- 
lieve the excess energy. Fripp adds, “Find a private spot, and 
wave your hands in the air. Relax your jaw, and shake your head 
from side to side. Then shake your legs one at a time. Physically 
shake the tension out of your body.” The object is to release 
enough nervous energy to calm your anxieties — without becom- 
ing so stress-free that you forget your purpose and audience. 


Watching yourself on video for the first time can be traumatic. In itself, the experience should 
help you overcome many problems. But if you are serious about improvement, ask the observers for 
feedback on your performance as you watch the video together. To get honest feedback, stick with 
open-ended questions such as the following: (1) Which supporting evidence was most effective? Why? 
Which supporting evidence was ineffective? Why? (2) Did the order of findings or arguments help 
support the conclusion? Why/Why not? (3) What was the most powerful element in the presentation? 
Why? (4) What would improve the presentation?*’ 


Controlling Performance Anxiety 


Performance anxiety, or stage fright, is a fear produced by the need to make a presentation in front 
of an audience or before a camera. In public speaking, it arises in anticipation of a performance or ac- 
companies the event and causes negative effects in presentational quality. In extreme cases, the fear is 
persistent phobia, which makes the individual dysfunctional in a presentation. 
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Performance anxiety has numerous physical symptoms: fluttering or pounding heart, tremor in the 
hands and legs, stomach cramping or nausea, facial nerve tics, flushing, hives, and dry mouth. More 
than 56 million citations on a Google search of “fear public speaking” give credence to the pervasive- 
ness of such anxiety. Performance anxiety at various levels occurs to people of all experience levels 
and backgrounds, from students to seasoned professionals.*® 

Research found five causal common denominators among individuals who experience performance 
anxiety—all based in negative self-perception: 


1. 


I perceive or imagine the presence of significant others who are able to judge me. 


2. I consider the possibility of my visible failure at a task. 
3. I feel a need to do well to avoid failure. 

4. 
5 


. I focus on my own behavior and appearance.*? 


I feel uncertain as to whether I will do well. 


Exhibit 20-14 Addressing Performance Anxiety 


Suggested Strategy Anxiety Reducing Actions 


Reduce the imagined power of e Remind yourself that you know the methodology and the findings far better than anyone in the 


others. 


audience. 


e Remind yourself that you have new information and new insights that could help resolve the 
manager’s problem. 


e See yourself as the audience’s partner in solving their problem. 


e Wear clothing that increases your power (suits win out over causal apparel). 


Eliminate imagining negative e Remind yourself of the positive outcomes of the sponsor adopting your recommendations... 


possibilities. 


their company grows, avoids layoffs, etc. 
e Plan for contingencies 
e Create a disaster kit with extra power cords, projection bulbs, and laptop. 
e Burn your presentation to CD, as well as to a USB thumb drive. 


e Make multiple copies of your script note cards or slide note pages, put them in different 
places (luggage, backpack, car). 


e Have multiple copies of handouts of your slides as a backup to a PowerPoint malfunction. 


Hold the performance in e Think of the presentation as an opportunity for career-enhancing experience. 


perspective. 


e Remind yourself of what you'll be doing later today or tomorrow that will provide you great joy. 
e Plan a dinner with friends the evening following the presentation. 


e Plan acelebration with your teammates for after the presentation. 


Control your own performance. e Get some exercise to burn off your nervous energy. 


e Eat a couple of hours before you go onstage to avoid low blood sugar (can make you feel 
light-headed) or too much undigested food (can make you nauseous). 


e Craft your support materials with great care. 

e Develop strong examples, exercises, slides, and handouts. 
e Practice, Practice, Practice. 

e Apply the visualization techniques that the professionals use. 


e Rest shaking hands on the podium to hide trembling. 


Increase your awareness of e Meet your audience (all or at least some) before your presentation. 


others without considering 


them judges. 


e Learn something personal about a few audience members that makes them appear more 
human . . . they have kids who eat bark, they like cherry Kool-Aid, they hate sunshine (or 
snow), they have a chihuahua named Brutus, etc. 
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The objectives of the meeting, the 
room, the technology, and the size and 
composition of the audience should all 
significantly influence the researcher's 
oral presentation. 


According to the authors, optimal strategies for coping with performance anxiety include (1) re- 
duce the imagined power of others by increasing the sense of one’s own power; (2) eliminate the 
imagination of negative possibilities, and think about the positive outcomes of a successful presenta- 
tion; (3) hold the performance in perspective by seeing its outcome as insignificant in relation to the 
totality of one’s life; (4) remember that one cannot control other’s reactions or judgments, but only 
one’s own performance; and (5) refocus one’s attention away from self and increase one’s awareness 
of others, without considering them as judges. In short, focus on “process rather than results, the 
moment of experience rather than the future, positive approach goals rather than negative avoidance 
goals, and self-acceptance rather than self doubt.”® See Exhibit 20-14 for strategies to reduce per- 
formance anxiety. 

Some readers will say, I understand the psychology, but what can you suggest that is really practi- 
cal? The best thing you can do is to be overwhelmingly, painstakingly, and totally prepared. After that, 
exercise. Presenters need regular exercise to reduce tension and stress. Desensitize with relaxation 
techniques (meditation, yoga, Tai Chi, Qi Gong, or EFT, as many entertainers do). If possible, talk with 
a few audience members before you speak to build your confidence and audience rapport. Before the 
presentation, or while being introduced, sit calmly and breathe slow, deep breaths. 


Arrangements for Facilities and Equipment 


Arrangements for the presentation occasion and venue, sometimes referred to as staging, involve 
detailed management of facilities, operational problems, and equipment (lecterns, lights, projectors, 
cords, controls, sound systems, video, Internet conferencing, electronic boards, racks for charts, dis- 
plays/models/props). Staging requires attention to the meeting room, seating arrangement, screens and 
lighting, testing of virtually everything, along with preparation/backup for disasters. Refer to Exhibit 
20-15 for a detailed checklist of the activities involved in perfecting the arrangements. 
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>Exhibit 20-15 Facilities and Equipment Checklist A and B 


Checklist A 


Source Considerations 


Q) Facilities Meeting room On-site vs. off-site 


Adjacent facilities and noise 
e Plain walls: distraction avoidance 


© Clock placement 


Ingres-egress opposite speaker 
e Barriers between presenter and audience 


Lighting Rheostats 
¢ Screen proximity and wash-out 


e Access to bulbs and fixtures 


Electrical power © Outlets: location 


¢ Power extensions 


Lectern e Moveable vs. fixed vs. podium 
e Location and visibility 
© Size adequate for presenter’s equipment 


Temperature e Adjustable vs. central 


Effect on audience 


Seating e Theater vs. conference style 


Conference table for small group: about 10 to 15 


Individual tables for larger group: 5 to 6 per table 


e U-shape for visibility and interaction 


(1 Sound System Microphone Need for professional sound specialist 
e System control access 
e Handheld: 6 to 10 inches vertical from chin 


e Fixed-handheld vs. wireless 


Feedback proximity 


Desirability of portable systems 
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Exhibit 20-15 Facilities and Equipment Checklist A and B (Continued) 


Checklist B 


Source 


Considerations 


(1 AV Equipment LCD projector 


Video 


Video conferencing/Webinars 


Flipcharts/posters 


Electronic whiteboards 


Portable vs. installed in room 
Projector-PC compatibility 
Computer power 

Operating location in room 
Wireless controller/mouse 


Wireless keyboard (meeting-related) 


DVD/camcorder/VCR 
Web streaming 
Teleprompter 
Playback monitor size 
System tests 


PC vs. Mac requirements 

VoIP 

Speaker systems 

One broadcast vs. two-way interaction 
Supported browsers 


Size and visibility 
Support systems—racks 


Simulation of PC desktop 
Create video files 

Digital story telling 
Brainstorming 

Port over to PowerPoint 
Use for review/repetition 


Source: adapted in part from Thomas Leech, How to Prepare, Stage, and Deliver Winning Presentations (New York: 


AMACOM, 2004), pp. 167-87. 


summary 


1 An oral research presentation has unique characteristics 
that distinguish it from public speaking but with which it 
shares similarities. A small group of people is normally in- 
volved; statistics often constitute an important portion of 
the topic; the audience members are usually managers with 
an interest in the topic, but they want to hear only the data 
and conclusions that will help them make critical decisions; 
speaking time will often be as short as 20 minutes, but may 
run longer than an hour; and the presentation is normally 
interspersed with questions and discussion. 


2 Aristotle’s rhetorical influence on business presentation in 
the 21st century acquaints us with persuasive possibilities in 
every presentational situation. The basis of persuasion has 
three principles of proof: ethos, pathos, and logos. Our per- 
ception of a presenter’s character affects how believable or 
convincing we find that person. This is called the speaker’s 
ethos. Pathos relies on an emotional connection between 
the speaker and his or her audience. It is an appeal to an 
audience’s sense of identity, self-interest, and emotions. 
With logos, the logical argument, we find the explicit reasons 
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that the speaker needs to support a position. This translates 
into supporting evidence and analytical techniques that re- 
veal and uphold the researchers’ findings and conclusions. 


We start to plan for the research presentation by accentuat- 
ing the audience’s role: Who is the audience? What do they 
want to learn? Why is this presentation occurring? When will 
it occur? Where will the presentation take place? The most 
important question is about the audience’s mind-set: Why 
should | care? Demographic and dispositional characteristics 
of the audience also play a role. An audience is composed of 
three types of learners: visual, auditory, and kinesthetic —all 
with different needs and learning styles. To keep the audience 
attentive, research presenters should observe the 10-minute 
rule by varying their content. They should also understand 
that arguments presented first or last will be highly influential 
to understanding and motivate others to take action. 


Presentations have a wide variety of organizational struc- 
tures that the presenter can use to construct a framework, 
thereby assisting the audience to follow and understand the 
presentation. Numerous traditional patterns of organiza- 
tion (topical, spatial, classification, climax order, problem/ 
solution, chronological, past/present/future, cause/effect/ 
solution, pros/cons/recommendation, and research brief- 
ing) are viable options, although the motivated sequence, 
the narrative style of development, the rule of three, and the 
three-point speech are preferred organization strategies. 


Supporting materials are leaves on the branches of the 
organizational framework. They include facts, statistics, spe- 
cific instances, examples, testimony/expert opinion, anal- 
ogy and metaphor, conveying personal experience through 
story, and demonstrations. 


Proficiency in research presentation requires creating good 
visuals and Knowing how to use them. The culprit of visu- 
ally poor presentations is often presenters who create and 
organize their message with PowerPoint or other design 
software (which disrupts, dominates, and trivializes content) 
rather than in analog, away from the computer. Visualization 
involves developing and organizing supporting materials that 
help the audience understand your findings. Several psycho- 
ogical principles influence the visualization of your presenta- 
tion: the principles of relevance (only information critical to 
understanding should be presented), appropriate knowledge 
limitations in your audience’s knowledge level), capacity 
imitations (inability to process large amounts of informa- 
tion at one time), informative changes (convey what is new 
with a separate slide or handout), salience (the audience’s 
attention is drawn to large perceptible differences), discrim- 
inability (two properties must differ by a large amount to be 
discerned), and perceptual organization (when establishing 
associations or correlations between key findings, your sup- 
porting materials should group in proximity to each other). 

In addition to selecting the right visual elements to con- 
vey your findings and conclusions, design principles should 
guide the design of visual support materials. They include 


visual preparation, flow aids, visibility, whitespace, picture 
supremacy, contrast, compatibility, relationship, simplicity, 
and clarity. 


How the speaker delivers the message is very important. 
Demeanor, posture, dress, and total appearance should 

be appropriate for the occasion. Speed of speech, clar- 

ity of enunciation, pauses, and gestures all play their part. 
Voice pitch, tone quality, and inflections are proper subjects 
for concern. Rapport-developing techniques are essential 
so that the speaker can get and hold the audience’s at- 
tention. The modes of delivery by which you communicate 
your message to the audience include impromptu speak- 
ing, memorization, manuscript reading, or extemporaneous 
presentation. We rule out the impromptu briefing, because it 
does not involve preparation, and discourage memorization 
and manuscript reading due to lack of audience connection. 
Scripts have no place in research presentations, where audi- 
ence members want to engage in information exchange, but 
are important in the planning phase. Audiences accept the 
use of speaker notes, which are consistent with extempora- 
neous presentations. 

Details make a difference in effective delivery, including 
reducing clutter, which gives the impression of hesitancy 
and lack of competence (repetition of fillers such as “ah,” 
“um,” “you know,” “like,”). Using jargon is a particular danger 
when your audience may not be schooled in the techniques 
of research or of statistical analysis. Jargon adds clumsy lan- 
guage and reduces the simplicity of the message. Nonverbal 
communication accounts for approximately 50 to 93 percent 
of communication meaning and, because it is sometimes 
more believable than verbal communication, deserves care- 
ful attention. Major categories include eye contact, gestures, 
posture and body orientation, and paralanguage. 


Practice is the critical ingredient that super achievers and 
star performers have in common. It serves to recreate the 
presentational setting as you practice, prevent embarrass- 
ment later, check your ratio of material versus available time, 
uncover holes in your supporting material, and prepare for 
the unexpected. A video rehearsal helps overcome many 
problems, especially with feedback from your practice audi- 
ence. Stage fright occurs to people of all experience levels 
and backgrounds, from students to seasoned professionals. 
While we suggest several means of coping, the best thing 
you can do is to be overwhelmingly, painstakingly, and to- 
tally prepared. 

Arrangements for the presentation occasion and venue, 
sometimes referred to as staging, involve detailed manage- 
ment of facilities, operational problems, and equipment 
(lecterns, lights, projectors, cords, controls, sound systems, 
video, Internet conferencing, electronic boards, racks for 
charts, displays/models/props). Furthermore, arrangements 
require attention to the meeting room, seating, screens and 
lighting, testing of virtually everything, along with prepara- 
tion/backup for disasters. 


analogy 553 

audience analysis 544 
auditory learners 546 
clarity 561 

contrast 558 
compatibility 559 

clutter 564 
demonstration 554 
enthymeme 542 

ethos 542 

example 552 

expert opinion 553 
extemporaneous presentation 562 
eye contact 565 

fact 552 

flow aids 557 

gestures 565 
impromptu speaking 562 
jargon 564 

kinesthetic learners 546 


logos 542 


Terms in Review 


1 Distinguish between the following: 
a_ Impromptu speaking and an extemporaneous presentation. 
b The motivated sequence and the cause/effect/solution 


pattern of organization. 


c The rule of three in organizing and the rule of thirds in 


visualizing. 
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manuscript reading 562 

memorization 562 

metaphor 553 

motivated sequence 549 

narrative pattern 549 

nonverbal communication 564 
paralanguage 565 

pathos 542 

performance anxiety 567 
photographic framing 558 

picture supremacy 558 

posture and body orientation 565 
primacy effect 546 

principle of appropriate knowledge 555 
principle of capacity limitations 555 
principle of discriminability 557 
principle of informative changes 557 
principle of perceptual organization 557 
principle of relevance 555 

principle of salience 557 


recency effect 546 


relationship 559 
research briefing 549 
rule of three 551 

rule of thirds 558 

script 562 

simplicity 559 

speaker note cards 563 
specific instance 552 
statistics 552 

stories 553 

10-minute rule 546 
testimony 553 
three-point speech 551 
visibility 557 
visualization 555 

visual learners 545 
visual preparation 557 
Web-delivered presentation 547 


whitespace 558 


Making Research Decisions 
4 The day before your presentation, you are suffering 


from pounding heart, a slight tremor in your hands, and 
stomach problems. What specific measures can you 


take to build your confidence and reduce performance 


anxiety? 


5 Outline a set of visual aids that you might use in an oral 


presentation on these topics: 


d Clutter and jargon in the delivery of a presentation. 


2 Describe the differences among logos, ethos, and pathos 
and their uses for the research presentation. 


3 What are the three types of learners, and how is the presen- 
tation different for each of these groups in your audience? 


a How to write a research report. 
b The outlook for the economy over the next year. 


c Amajor analytical article in the latest issue of 
BusinessWeek. 
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>part IV _ Analysis and Presentation of Data 


Your class team in research methods has completed a 
field project for a financial institution on branch location 
effectiveness. What questions about audience analysis 
should you answer as you plan your presentation? 


Bringing Research to Life 


7 


Every presentation has its purpose. What was the purpose 
of Henry & Associates presentation to MindWriter? 


How did the Henry & Associates presentation: 
a Perform an audience analysis? 


b Engage the audience? 


From Concept to Practice 


9 
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11 


In your presentation to a venture capital company, you are 
pitching your research plan for a smartphone application 
that does short-term cash flow forecasts. Describe how 
you would use each type of supporting material to create 
interest, clarify your point, provide emphasis to a point, and 
offer proof that results in belief. 


Using Exhibit 20-9, choose an appropriate graphing tech- 

nique to show the difference between men’s and women’s 
attitudes on the top five Super Bowl ads. (You can find this 
listing on http://adbowl.com.) 


You are creating slides for a presentation that is being 
given to seek a research contract with a leading toy 
manufacturer. Considering the following aspects of your 
presentation (introduction, problem to be solved, market 
opportunity, technology, manufacturing/production, finan- 
cials, conclusion), create slides representing four of these 
areas using Exhibit 20-13. 


From the Headlines 


12 


13 


You are preparing to give a research presentation about 
the effectiveness of Toyota’s advertising to restore 
public confidence in the wake of their delays in solving 
the accelerator pedal malfunction and antilock braking 
problems for high-tech hybrid vehicles. Which of the pat- 
terns of organization would be appropriate for your pur- 
pose? Why? 


The Casey Anthony murder trial garnered headlines 
around the world, not just for the gavel-to-gavel news 
coverage it received but for the ‘not guilty’ verdict on 
murder for the death of her daughter, Caylee. Casey was 
also found not guilty on aggravated child abuse, and 
aggravated manslaughter of a child. These verdicts were 
delivered after the presentation of more than 400 pieces of 
evidence and 90 witnesses during more than 33 days of 
testimony. The jurors indicated that some of the evidence 
was irrelevant to their discussion and decision, while oth- 
ers were not compelling. A prosecutor must choose from 
the evidence collected what they will present to the jury, 
much like a researcher must make choices when present- 
ing findings. And both must present that evidence in a way 
that the correct conclusions can be drawn. Public opinion 
differed significantly from the jury’s verdict. What presen- 
tation guidelines can be gained for researchers from this 
headline story? 
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*You will find a description of each case in the Case Index section of this textbook. Check the Case Index to determine whether 
a case provides data, the research instrument, video, or other supplementary material. Written cases are downloadable from the 
text website (www.mhhe.com/cooper1 2e). All video material and video cases are available from the Online Learning Center. 
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LEGEND: 
W = Written Case | = Instrument 
D = Data File Q = Some Measurement Instruments 


T = Some Table/Graphed Data 


VC = Video Case 
V = Some Video Material 


A GEM of a Study W, T Xx 

AgriComp W, D x 

Akron Children’s Hospital W, V Xx Xx x x 

Calling Up Attendance W, | Xx Xx 

Campbell-Ewald Pumps Awareness into the American Heart Association W, V Xx Xx 

Campbell-Ewald: R-E-S-P-E-C-T Spells Loyalty Ww Xx 

Can Research Rescue the Red Cross? WwW X 

Covering Kids with HealthCare VC X Xx 

Cummins Engines VC Xx 

Data Development, Inc. VC Xx 

Donatos: Finding the New Pizza W, V x x 

Envirosell VC 

Goodyear’s Aquatred VC Xx Xx Xx 

HeroBuilders.com WwW Xx X 

Inquiring Minds Want to Know—Now! W, | X Xx Xx Xx X Xx X X 
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Ohio Lottery: Innovative Research Design Drives Winning W, Q,T, V Xx Xx Xx Xx Xx Xx Xx 
Open Doors: Extending Hospitality to Travelers with Disabilities Ww Xx 

Pebble Beach Co. VC Xx 

Proofpoint: Capitalizing on a Reporter’s Love of Statistics W,T Xx Xx Xx Xx Xx Xx Xx 
Ramada Demonstrates Its Personal Best™ Ww Xx Xx 

Starbucks, Bank One, and Visa Launch Starbucks Card Duetto™ Visa VC Xx Xx Xx 

State Farm: Dangerous Intersections Ww Xx Xx Xx 

The Catalyst for Women in Financial Services W,T x 

USTA: Come Out Swinging W, |, VC Xx Xx 

Volkswagen's Beetle VC Xx X 

Yahoo!: Consumer Direct Marries Purchase Metrics to Banner Ads WwW Xx 


>case index 


> A GEM of a Study 


The Global Entrepreneurship Monitor Entrepreneurial Assessment, a joint project of the Kauffman 
Center for Entrepreneurship Leadership at Babson College and the London Business School, has un- 
dertaken a long-term, large-scale project to prove the causal links between a government’s economic 
policies and initiatives, the resulting entrepreneurial activity, and subsequent economic growth. This 
case describes multiple-stage research, including thousands of interviews in several countries by estab- 
lished research firms. www.babson.edu; www.I|sbf.org.uk; www.gemconsortium.org 


> AgriComp 


AgriComp, a supplier of computer systems for farmers, has surveyed its dealers on whether to change 
its procedure for settling warranty claim disputes. Currently local dealers handle warranty services for 
customers via local repair followed by a reimbursement claim to AgriComp. Denied claims follow an 
internal company appeal process. Dealers have been complaining about the fairness of the appeal pro- 
cess and in a recent survey were asked to respond to an alternative process, an impartial mediator. The 
student is asked to review survey results and determine whether the costly external mediator process 
would be worth implementing to keep the dealers happy. 


> Akron Children’s Hospital 


Northeastern Ohio is a highly competitive health care market, especially for the care of seriously ill chil- 
dren. With powerhouse health care institutions like the Cleveland Clinic venturing into the children’s care 
segment, Akron Children’s needed a way to differentiate itself. The research profiled in this case helped 
develop the positioning of Akron Children’s hospital and its promotional approach and resulted in an 
increase in its bed-occupancy rate, a key metric in the health care industry. www.akronchildrens.org; 
www.marcusthomasllc.com 


> Calling Up Attendance 


This case examines a study by Prince Marketing for TCS Management Group. TCS Management 
Group, Inc., part of Aspect Communications, is the leading provider of workforce management soft- 
ware, especially related to call center management. The study discusses measures of customer satis- 
faction and aims to predict attendance at a two-day educational event, Users Forum. www.aspect.com 


> Campbell-Ewald Pumps Awareness 
into the American Heart Association 


You wouldn’t think that an organization that does as much good as the American Heart Association 
would have low awareness, but at the start of the described research program its unaided awareness 
level was just 16 percent. For a company reliant on contributions, low awareness is a major problem. 
This case profiles the research behind the American Heart Association’s first-ever paid advertising 
campaign. Www.c-e.com; Www.americanheart.org 
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>case index 


> Campbell-Ewald: R-E-S-P-E-C-T Spells Loyalty 


Campbell-Ewald, the Detroit-based marketing communications company, part of the global Interpublic 
Group of Companies, is an award-winning consultancy. This case describes the research behind its 
effort to measure and improve customer loyalty and the development of its five respect principles that 
lead to enhanced customer commitment. www.c-e.com 


> Can Research Rescue the Red Cross? 


The American Red Cross seemed in its true element following September 11, 2001. It was flooded with 
donations to do its highly needed and regarded work. Most of those donations went to its Liberty Fund. 
But shortly after it started to disperse the funds, the media began asking questions. And the American 
Red Cross soon wore a patina of tarnish. Learn about the research that evaluated Americans’ percep- 
tion of the Red Cross and how research by Wirthlin Worldwide helped craft a new and highly effective 
donation solicitation process. www.wirthlin.com; www.redcross.org 


> Covering Kids with Health Care 


This video case describes the research done to increase enrollment in the federal government’s SCHIP 
program. Managed at the state level, the State Children’s Health Insurance Program provides basic 
health coverage for the children of the nation’s working poor. Research by Wirthlin Worldwide re- 
vealed why families weren’t enrolling, and findings were used by GMMB, Inc., to develop a major 
advertising and public relations initiative to increase enrollment. The research and campaign were 
sponsored by the Robert Wood Johnson Foundation. (Video duration: 16 minutes) www.rwjf.org; 
www.wirthlin.com; www.gmmb.com 


> Cummins Engines 


Cummins Engines makes advanced, fuel-efficient diesel power systems and engine-related components 
and specializes in customized diesel engine production. Shipping more than 1,000 engines per day to 
customers and dealers on every continent, Cummins has a long history of innovation, from winning 
performances at the Indianapolis 500 to the first natural gas—fueled engine to pass California’s tough 
emissions regulations. This case focuses on the Signature 600 engine, the newest and most advanced 
diesel engine on the market. (Video duration: 14 minutes) www.cummins.com 


> Data Development Corporation 


This video case profiles Data Development Corporation (DDC), a leader in in-home and office personal 
interviewing. DDC has fielded more than 20,000 studies since 1960; it currently has four offices in 
the United States, with global capabilities in 80 countries worldwide. DDC WATS centers have 170 
CATI (computer-assisted telephone interviewing) stations. The company offers a network of CAPI 
(computer-assisted personal interviewing) in more than 180 mall locations, as well as interactive soft- 
ware (STORE) simulations of store shelving, buildings, and so on, to develop and evaluate logos, 
signage, packaging, and the like. DDC’s Internet Survey Group offers Web-based studies. (Video 
duration: 11 minutes) www.datadc.com 


>case index 


> Donatos: Finding the New Pizza 


The pizza segment of the fast-food industry is very aggressive. As people’s tastes change and new diets 
become the rage, restaurant chains must decide if and how to respond. This case focuses on the research 
behind the introduction of Donato’s low-carbohydrate pizza and how the company collapsed its normal 
product-development research process to take advantage of a current trend. www.donatos.com 


> Envirosell 


Envirosell specializes in behavioral research, specifically in the retail environment. It has done this 
for Fortune 500 companies including banks, stores, and restaurant chains, as well as consumer prod- 
uct companies. In this video case, the managing director, research director, and senior analyst share 
information from several observational studies done in banks, as well as music, general-merchandise, 
and other retail environments. Envirosell, which has offices in the United States, Europe, Brazil, Japan, 
Mexico, and Turkey, strives to understand what people buy and how to get them to buy more. (Video 
duration: 10 minutes) www.envirosell.com 


> Goodyear’s Aquatred 


This video case profiles the genesis of the Goodyear Aquatred tire. In 1993, the Aquatred tire, winner 
of more than a dozen awards, including Japan’s prestigious Good Product Design Award, reached more 
than 2 million units in sales in the United States. This revolutionary tire pumps away more gallons of 
water per second as you drive at highway speeds. And a new tread rubber compound provides road- 
hugging traction and extends the tread life. The Aquatred tire segmented the market in a way that had 
not been done before. (Video duration: 14 minutes) www.goodyear.com 


> HeroBuilders.com 


Emil Vicale, president of BBC Design Group, used rapid prototyping technology (RPT) to build wax 
or plastic three-dimensional prototypes of his clients’ designs. But this same technology can be used 
to custom-manufacture dolls. Shortly after September 11, 2001, Vicale Corporation, BBC’s parent 
company, purchased an e-commerce toy company. Vicale’s first action figure was made to honor the 
heroes who emerged from that event. Using RPT, he crafted a doll with the head of George W. Bush 
and the body of Arnold Schwarzenegger. Other figures followed. This case is about a design firm that 
used exploratory research to define a niche in the action-figure business. www.herobuilders.com 


> Inquiring Minds Want to Know— NOW! 


This case describes a multistage communication study undertaken by the research department of Penton 
Media, a publisher of business trade magazines, to determine the long-term viability of a reader and 
advertiser service, the reader service card, a postcard-size device used by readers to request additional 
information from a particular advertiser. www.penton.com 


> Lexus SC 430 


This video case follows the research used to develop the newest Lexus, the SC 430, the line’s hardtop 
convertible. From auto show interviews to qual-quant clinics and positioning analysis, learn about 
how Team One Advertising, Lexus’s U.S. agency of record, used research to position this latest 
entry into the crowded sport coupe category. (Video duration: 8 minutes) www.teamoneadv.com; 
www.lexus.com 
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>case index 


> Marcus Thomas Tests Hypothesis for Troy-Bilt® 


Creative Development 


Troy-Bilt® works with Marcus Thomas LLC to develop marketing communications, including television 
advertising campaigns. As part of its creative development, Marcus Thomas needed to understand what 
motivated the yardwork enthusiast, and what makes television advertising most effective within the lawn 
and garden industry. Marcus Thomas developed a hypothesis that “consumers who are in the market 
to purchase a product process television advertising differently than those who are not in the market 
for lawn and garden equipment.” It used an online survey employing a control group, with embedded 
video, to test this hypothesis and develop the subsequent ad campaign. www.marcusthomasllc.com; 
www.troybilt.com 


> Mastering Teacher Leadership 


This case is about a multistage communication study of teachers by Wittenberg University’s Depart- 
ment of Education to determine the viability of starting a Master of Education program for Ohio- 
certified teachers working within school districts serving a five-county area. www.wittenberg.edu 


> McDonald’s Tests Catfish Sandwich 


This case describes the test marketing for McDonald’s catfish sandwich in the southeastern United 
States. It asks students to assume they are the new product development team and to assess the research 
design described. www.mcdonalds.com 


> NCRCC: Teeing Up a New Strategic Direction 


The NCR Country Club started out as a benefit for thousands of National Cash Register employees. By 
the late 1990s, those employees were aging rapidly and the core membership needed to be increased. 
NCRCC offers two golf courses. One is an award-winning, championship-hosting course on the PGA 
tour. But the club wasn’t attracting new members, especially younger families. This case is about a 
membership study done as part of a larger management initiative to evaluate several strategic directions 
the club might take to expand its membership. www.ncrcountryclub.com 


> NetConversions Influences Kelley Blue Book 


Kelley Blue Book (KBB) is one of the most visited automotive sites on the Web. Visitors flock there 
to estimate the price of a car they might buy or sell. KBB needed to enhance its site’s performance for 
advertisers, who had become a major source of revenue as sales of the printed Kelley Blue Book had 
declined. NetConversions is one of the new Web analytic services to evaluate website performance. 
This case reveals how websites are evaluated so that new design elements can be developed and tested. 
www.netconversions.com; www.kelleybluebook.com 


>case index 


> Ohio Lottery: Innovative Research 
Design Drives Winning 


The Ohio Lottery was originally developed as an additional source of public school funding. Today 
proceeds from lottery games annually provide approximately 7 percent of the public educational bud- 
get. This research was originally undertaken because the lottery director wanted a deeper understanding 
of lottery players and insight into nonplayers. The research design described in this case is multistage 
and incorporates the use of both qualitative and quantitative research. This case reveals the research 
that guides the current Ohio Lottery promotional program that encourages play of its various games. 
www.marcusthomasllc.com; www.mrsi.com; www.ohiolottery.com 


> Open Doors: Extending Hospitality 
to Travelers with Disabilities 


Eric Lipp started the Open Doors Organization (ODO) to help travelers with disabilities. In order to 
get the attention of the travel and hospitality industries, and to effect changes desired by people with 
disabilities, ODO undertook a major research project to estimate the expenditures of persons with dis- 
abilities and the accommodations that would be necessary to get them to travel more. Harris Interactive 
was chosen to field the multimethod survey. This case describes the methodology and the effects of the 
first round of a multiphase study. www.opendoorsnfp.org 


> Pebble Beach Co. 


This case profiles the Pebble Beach Company, a 5,300-acre complex in Monterey, California, that 
offers three lodging options (Casa Palmero opened in September 1999, The Inn at Spanish Bay opened 
in 1989, and the Lodge at Pebble Beach opened in 1919), four golf courses, plus a new five-hole “golf 
links,” eight restaurants, and an oceanside Beach and Tennis Club. Pebble Beach has repeatedly won 
awards as America’s best travel resort and was host to the 2004 AT&T Pro-Amateur championship, the 
U.S. Open in 2000, the Callaway Golf Pebble Beach Invitational in 2004, and the newest tournament 
on the 2005 PGA Champions tour (First Tee Open). In January of 1999 The Inn at Spanish Bay was 
granted the coveted Mobil Five-Star Award from the 1999 Mobil Travel Guide. Pebble Beach achieves 
its quality status by focusing on seven core values. The company is land-locked, so it must develop 
ever-creative ways to make the facilities it has more intensively profit-generating. (Video duration: 
11 minutes) www.pebblebeach.com 


> Proofpoint: Capitalizing ona 
Reporter’s Love of Statistics 


Proofpoint provides antispam software and e-mail security software solutions for large enterprises. 
Their software products stop spam, protect against e-mail viruses, ensure that outbound e-mail mes- 
sages comply with both corporate policies and external regulations, and prevent leaks of confidential 
information via e-mail and other network protocols like blogs and text-messaging. Proofpoint knew 
from its customer inquiries that the IT professional was increasingly concerned with outbound infor- 
mation privacy compliance issues. Believing that painting this broader picture would earn them valu- 
able space and airtime in the business, IT, and mainstream media, Proofpoint sponsored a series of 
surveys among IT professionals. This case is about those surveys. www.proofpoint.com 
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>case index 


> Ramada Demonstrates Its Personal Best™ 


This case describes syndicated research in the hospitality industry that revealed trends in customer 
satisfaction and Ramada’s proprietary research leading to the development of the Personal Best™ 
employee hiring, training, and motivation program. www. ramada.com 


> Starbucks, Bank One, and Visa Launch 


Starbucks Card Duetto™ Visa 


In the very mature financial services industry, it is rare for a new financial product to garner much at- 
tention, let alone be named one of BusinessWeek’s outstanding products of the year. But what started 
as a way for Starbucks to add value to its existing Starbucks Card program developed into a financial 
product that many other institutions are interested in exploring. This case reveals the research that 
was done to develop this new payment option for Starbucks customers. (Video duration: 11 minutes) 
www.starbucks.com; www.bankone.com; www.visa.com 


> State Farm: Dangerous Intersections 


State Farm, the nation’s largest auto insurer, distributed a list of the 10 most dangerous intersections 
in the United States based on crashes resulting in claims by its policyholders. What started as a study 
to reduce risk turned into an ongoing study that directs a major public relations effort: State Farm 
provides funds for communities to further research their dangerous intersections and initiate improve- 
ments based on the research. This case tells how the State Farm Dangerous Intersections initiative got 
started and how it is done. www.statefarm.com 


> The Catalyst for Women in Financial Services 


Smith Barney (now Morgan Stanley Smith Barney) was ordered by the court, in settling the landmark 
sexual harassment case, to evaluate the climate for personal development and advancement through 
promotion not only in Smith Barney but, as a comparative, in other financial services firms. The case 
describes the methodology used to sample both men’s and women’s beliefs and attitudes, as well as 
reveals some of the basic findings. www.catalyst.org; www.smithbarney.com 


> USTA: Come Out Swinging 


The United States Tennis Association funded one of the most aggressive surveys ever undertaken 
about a single sport in order to revitalize tennis in the minds of consumers. The survey results were 
supplemented with qualitative research by Vigilante, a specialist in urban communication campaigns. 
What resulted was a full-scale marketing initiative involving the establishment of Tennis Welcome 
Centers and the Come Out Swinging advertising, merchandising, and public relations campaigns. This 
case reveals the research and how the marketing initiative developed. (Video duration: 11 minutes) 
www.usta.com; www.vigilantenyc.com; www.thetaylorgroup.com 


>case index 


> Volkswagen’s Beetle 


This video case profiles the history of the original Beetle in the U.S. market from its introduction in 
1949 to its demise in 1979 and then follows the initial two years of the New Beetle’s rebirth, 1998 
and 1999. The Beetle became a symbol of the 1960s rebelliousness, but it lost the love of a generation 
when it stressed engineering over style and low-cost operation, two factors that baby boomers consid- 
ered crucial in the 1970s. By 1974, the Beetle had lost ground to its aggressive Japanese rivals for the 
value segment of the U.S. automobile market. In 1998, when the Beetle was reintroduced in the United 
States, it surpassed all sales estimates. The second year it doubled its sales. Historically, the Beetle is 
the world’s best-selling car, having sold in more countries than any other automobile, with 21 million 
cars sold in its lifetime. (Video duration: 16 minutes) Www.vw.com 


> Yahoo!: Consumer Direct Marries 
Purchase Metrics to Banner Ads 


As little as two years ago, many advertising pundits were bemoaning the inevitable demise of the banner 
ad on the Internet. But maybe they were too quick to judge. This case reveals how Yahoo!, in combi- 
nation with ACNielsen’s Homescan®, has developed a methodology (Consumer Direct) to evaluate 
the true effectiveness of banner ads, from ad exposure to shopping cart. It also reveals the role 
Dynamic Logic played in conducting postexposure ad evaluation. www.yahoo.com; www.nielsen.com; 
www.dynamiclogic.com 
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Proposing Research 


Many students and some business researchers view the proposal process as unnecessary work. In ac- 
tuality, the more inexperienced a researcher is, the more important it is to have a well-planned and 
adequately documented proposal. The proposal process, Exhibit a-1, uses two primary documents: the 
request for proposal (RFP) and the research proposal. When the organization has research specialists 
on the payroll, the internal research proposal is often all that is needed. Often, however, companies do 
not have adequate capacity, resources, or the specialized talents in-house to execute a project, so they 
turn to outside research suppliers (including research specialists, universities, research centers, and 
consulting firms). We will explore the second scenario first. 


The Request for Proposal (RFP) 


The request for proposal (RFP) is the formal document issued by a corporate research department, 
a decision maker, or some other sponsor to solicit services from research suppliers. Developing a 
well-written RFP takes time and planning. However, the benefit to the sponsoring organization is an 
opportunity to formalize the process of documenting, justifying, and authorizing the procurement of 
research. RFPs also provide a chance to evaluate different solutions and offer the means of establish- 
ing, monitoring, and controlling the performance of the winning supplier. 

The researcher invites a qualified supplier to submit a proposal in accordance with a specific, de- 
tailed format—delivered by a deadline. Prescribing a common format makes comparison of competing 
proposals much easier. Each firm has its own requirements, and these are reflected not only in the form 
of the RFPs but in how they’ re distributed. The government, for example, is required by law to publicly 
announce RFPs. Private firms may limit supplier invitations to bidders that they have solicited before, 
to vendors that have provided past services, or to a single bidder (sole source). Both technical merit and 
the supplier’s estimate of project cost determine how contracts are awarded. 

Research suppliers consider RFPs an important source of future business. Thus, they must be vigi- 
lant to retain credibility with current and past clients and must seek to achieve positive word of mouth. 
Professional guides or business listing services promote the supplier’s visibility. Companies sometimes 
avoid the formal RFP as a means of contacting suppliers. They may invite you to propose a project dur- 
ing a conversation and later ask you to formalize it in writing. Moreover, not all projects are conducive 
to the RFP process. However, in the next section, we will discuss how an organization requests state- 
of-the-art proposals for dealing with complex research problems. 


Creating the RFP 


The first step is to define and understand fully the problem being addressed. In formal RFP processes, 
internal experts define the problem. They may be brand managers, new product specialists, or repre- 
sentatives from other functions. Alternatively, an expert or a group of experts may be retained to assist 
in defining the problem and then writing the RFP. In the tourism study, members of the commerce 
department and experts in the hospitality, travel, advertising, and entertainment fields would have par- 
ticipated at the request of the governor. Once the problem is defined, the technical section of the RFP 
can be written. 


>appendix a Business Research Requests and Proposals (with Sample RFP) 


>Exhibit a-1 The Research Proposal Process 


Decision 
without 
Research 


Cost exceeds 


Value exceeds 
cost 


Decision 
without 
Research 


Approve Budget 
and Design 


Budget or 
design rejected 


Proposal 
approved 
Proposal 
approved 


Award 
Research 
Contract 


Execute 
Research 


Besides a definition of the technical requirements of the desired research, critical components of the 
RFP include project management, pricing, and contract administration. These sections allow the potential 
research supplier to understand and meet the expectations of the sponsoring management team for the 
contracted services. Also, a section on proposal administration, including important dates, is included. 

An important activity that precedes this is qualifying potential suppliers. Sponsors must determine which 
vendors have the capability to complete the project on time. When the process is not open to all bidders, 
criteria such as industry experience, reputation, geographic location, quality of previous work, size of staff, 
and strategic alliances with other vendors determine which bidders will be eligible to receive the RFP. 

Although RFPs differ somewhat from firm to firm, the general components are: 


e Proposal administration information. 

e Summary statement of the problem. 

¢ Technical section. 

e Management section. 

¢ Contracts and license section. 

e Pricing section. 

Proposal Administration This section is an overview of important information on the administra- 
tion of the project itself. It establishes the dates of the RFP process—when the RFP is released, when the 
RFP team is available for questions, the date the proposal is expected, and the dates of the evaluation and 


supplier selections. It includes all requirements for preparing the proposal and describes how proposals 
will be evaluated. Contact names, addresses, and relevant telephone and fax numbers are listed. 
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>appendix a Business Research Requests and Proposals (with Sample RFP) 


Summary Statement of the Problem The summary statement can be an abstract of the technical 
section, or it can be included as the first page of the technical section. It often takes the form of a letter 
introducing the organization that issued the RFP and explaining its needs. As an example, let’s use a 
problem statement from MindWriter that deals with a customer satisfaction issue: 


The call center in MindWriter’s new CompleteCare facility currently operates without an automated 
recording and monitoring process. We have 10 dedicated reviewers sustaining this function. In ad- 
dition, our call center supervisors spend six hours per month monitoring the quality of our review- 
ers. The reviewers rely on a manually generated schedule to select representatives and times for 
monitoring. When representatives are on active calls during the monitoring schedule, it is problem- 
atic to trace them. However, reviewers have access to the online scheduling software. Thus, they 
can view the account screens selected by the representative using our own software tool. 

The quality of our customer service and the resulting satisfaction are of vital importance to 
MindWriter. We need to significantly increase the efficiency of our customer call monitoring through 
automation and a recorded database for agent review. We also need to discover the extent to 
which these technical changes to our process improve customer perceptions of service. 


Technical Section Technical information needed by the supplier to create the proposal is pre- 
sented in this section. It begins by describing the problem(s) to be addressed and the technical details 
of each requirement. It loosely describes the services to be performed and the equipment, software, and 
documentation required. This section should be neither too specific nor too general to allow the suppli- 
ers reasonable flexibility and creativity in research design but should also restrict them in meeting the 
needs of the sponsor. Typically, the following would be included: 


e Problem statement. 
¢ Description of functional requirements (what actual phases will be included in the research). 
° Identification of constraints (what might limit research design creativity). 


The sponsor’s functional requirements assist suppliers in testing the comprehensiveness of their pro- 
posed solutions. Often, sponsors ask the proposed researcher to answer questions. In MindWriter’s 
RFP, the writers have considered a wide range of functional issues: 


Recording 

e What proportion of calls does your proposed solution record? 
¢ To what degree is your proposed system scalable? 

¢ Can the representatives detect that they are being recorded? 


System Integration and Retrieval 
¢ Can you integrate multiple sources of information to the recording platform? 
e Does your proposed solution offer redundancy in the event of a failure? 


e Does your proposed solution store conversations along with their corresponding call-tag data ina 
single database? 


e Can the recorded calls be replayed immediately? 
e How does your proposed solution search calls for replay? 
e What volume of long-term archived storage is available? 


Evaluation and Analysis 

° Can call data be displayed visually for analysis? 

° How are calls selected for evaluation/scoring? 

¢ Can values be assigned to each question and “rep performance” or “rep skill” category? 
e Does your solution offer data mining capabilities? 


e In what ways does your solution support managerial analysis of operations and business 
performance? 


e How does your solution support the CompleteCare customer satisfaction philosophy at MindWriter? 


Strategies for dealing with constraints include specifying what is anticipated. If the sponsor requires that 
the supplier offer creative solutions, the RFP describes the constraints within which solutions must work. 
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A client of Jason Henry’s provides an example, below, of sampling constraints in its RFP. The client is 
interested in using the benchmarks from previous studies and thus needs consistency in its current project. 


° The sample sizes and breakdowns for various markets are: 
¢ Europe: 500 completed surveys. 
e Asia: 500 completed surveys. 
e United States: 300 completed surveys. 
° Regional differences. 


° Differentiation by segment and brand. 


° Proposed sample proportion for distributors/resellers: 
° Resellers = 90-95% of respondents. 
e Distributors = 5-10%. 


Building technical quality control into the RFP will subsequently strengthen the project. When the 
technical section contains thorough specifications and clear criteria for evaluating proposals, even low 
bidders must provide the requisite quality for consideration. In addition, when the RFP requires that 
the supplier provide technical reports during the project, project management is less costly for the 
firm. When a thorough understanding of the constraints is unknown, sponsors may schedule a planning 
meeting with possible researchers prior to their RFP response to clarify and examine options. 

Management Section Each project requires some level of management. The sponsor’s timing 
on schedules, plans, and reports is included in this section. The management section also lists the re- 
quirements for implementation schedules, training and reporting schedules, quality control, and other 
documentation. If specific supplier qualifications are needed, they should be shown here. References 
from the supplier’s customers may also be requested. Increasingly, detailed documentation websites 
are used to provide additional information to those invited to submit proposals. These website URLs 
are documented in the RFP. 

Contracts and License Section The types of contracts the supplier is expected to sign and any 
nondisclosure agreements are included in this section. The supplier of research is often privy to a 
firm’s strategies and tactics long before such competitive moves are undertaken. The supplier is also 
aware of challenges facing the firm and actions being considered to address those challenges. Non- 
disclosure of such information is therefore critical. It is in this context that the sponsor should discuss 
the safeguarding of intellectual property and the use of copyrights. Terms of payment and required 
benchmarks are also set forth here. Typically, a sample purchase contract would be included. Since the 
RFP document is usually a part of the final contract, it should be worded precisely to avoid problems 
of interpretation. If a task is not described in the RFP or during contract negotiations, the firm may not 
be able to require that the supplier complete it. 

Pricing Section To cost the proposal, suppliers must receive all needed information. A format that 
lists all anticipated activities helps the sponsor compare the cost of proposals with different approaches. 
The following list shows examples of items that could be included: 


e Services. ¢ Report preparations. 

e Data collection. ¢ Computer models. 

e Data analysis. e Project management. 

¢ Meetings with client. * Questionnaire and reproduction costs. 
¢ Travel. e Manpower costs. 

e Respondent survey incentives. ¢ Deliverables: 

e Mail and telephone costs. e Training. 

e Design meetings. ¢ Brochures/literature. 
e Internet design and activation. e Videotapes. 

¢ Facilities and equipment. ° Reports. 

e Extensions to work agreements. e Promotionals. 


e Pilot tests. 
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Ethical standards are integral to designing the pricing section. For example, a sponsor would not send a 
vendor an RFP to (1) help the sponsor plan its project budget, (2) estimate costs and ideas for a project 
the sponsor intends to execute in-house, or (3) create the impression of a competitive bid when the 
sponsor intends to sole-source the project. 

Format The format requirements for RFPs differ widely. The sections above reflect informational 
requirements rather than an RFP outline. A typical format might contain the following elements: 

¢ Instructions to bidders. 

¢ Background. 

e Overview or profile of the buyer’s company. 
e Project overview. 
e Project requirements. 
e Vendor information. 
e Company profile. 
e History and description. 
e Legal summary (active lawsuits or pending litigation). 
e Partnerships and alliances. 
e References. 

¢ Proposed solution. 

e Services and support. 

¢ Cost proposal. 

e Services pricing. 

e Maintenance pricing. 

¢ Contractual terms and conditions. 
Because each research project is often unique, industry practices suggest that careful consideration 
should be used when qualifying potential research suppliers. Exhibit a-2 offers a checklist developed 
from recommendations of industry practitioners and associations. 

To recap, the manager, research department, or research sponsor should achieve several objectives 
in the RFP process: qualify potential vendors, write and distribute the RFP eight to ten weeks before 
the requested date, be available to answer supplier questions or hold prebidding conferences, evaluate 
submissions on known criteria, award contracts and start the project on published dates, and provide 
a critique to all suppliers who submitted proposals. The latter will help unsuccessful bidders become 
competitive in the future and maintain your goodwill for future projects. 

A well-written RFP allows an organization to request high-quality proposals for dealing with com- 
plex problems. When not done properly, the RFP process will take longer, cost more, and not provide a 
complete long-term solution. Therefore, when a manager decides to put a research project to bid using 
an RFP, it is essential that time and effort be invested at the beginning. 

Now, let’s say as a researcher you have received an RFP. What is next? First, you decide if creat- 
ing a proposal is worth your investment of time and effort. Even if you are not responding to the RFP, 


becoming familiar with proposals can be helpful. As a researcher, you might consider producing all of 
your projects using a structure or template similar to the proposal format. 


The Research Proposal 


A proposal is an individual’s or company’s offer to produce a product or render a service to a potential 
buyer or sponsor. The purpose of the research proposal is: 

1. To present the management question to be researched and relate its importance. 

2. To discuss the research efforts of others who have worked on related management questions. 


3. To suggest the data necessary for solving the management question and how the data will be 
gathered, treated, and interpreted. 
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>Exhibit a-2 Checklist for Qualifying Research Suppliers 


(4 Research experience and industry status including appropriate accreditation. 


(1 Scope/type of research performed (quantitative vs. qualitative vs. both; advertising creative development, 
product testing, site location, etc.). 


(1 Knowledge of specific research methodologies (e.g. research with children, visual ethnography, 
conjoint analysis). 


(4 Types of clients. 
(4 Knowledge of specific markets. 
{4 International links or associations, if needed. 
{4No conflicts of interest. 


4 Code of ethical performance. 


OI Skill and experience to manage the project. 
O Skill and experience to conduct desired research. 
O Specialist skills, when needed (psychologists, anthropologists, Internet technologists, etc.). 


(J Understanding of the various business functions. 


“4 Compatible project management system. 
4 Compatible contractual arrangements, including billing. 
“1 Compatible client complaint and satisfaction handling procedures. 


11 Desired quality assurance procedures. 


“4 Desired organization, procedures, and appropriate facilities. 
Q Data collection (interviewers, interviewer training, CATI, CAPI, proprietary methodologies, etc.). 


1 Field operations. 


“1 Lab settings (taste testing, product testing, etc.). 


X Data handling (internal or subcontracted, software used, etc.). 


M4 Developing/drawing samples. 


“1 Compatible standard reporting procedures and guidelines. 


LI Desired results presentation practices. 


Source: This checklist was developed from recommendations of industry practitioners and material on the ESOMAR 
website: http://www.esomar.nl/guidelines/CommissioningResearch.htm. 


In addition, a research proposal must present the researcher’s plan, services, and credentials in the best 
possible way to encourage the proposal’s selection over competitors. In contract research, the survival 
of companies depends on their ability to develop winning proposals.' A proposal is also known as a 
work plan, prospectus, outline, statement of intent, or draft plan.” The proposal tells us what, why, how, 
where, and to whom the research will be done. It must also show the benefit of doing the research.? 

The research proposal is essentially a road map, showing clearly the location from which a journey 
begins, the destination to be reached, and the method of getting there. Well-prepared proposals in- 
clude potential problems that may be encountered along the way and methods for avoiding or working 
around them, much as a road map indicates alternate routes for a detour. 
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Sponsor Uses 


All research has a sponsor in one form or another. The student researcher is responsible to the class 
instructor. In a corporate setting, whether the research is being done in-house by a research department 
or under contract to an external research firm, management sponsors the research. University-, govern- 
ment-, or corporate-sponsored (grant) research uses grant committees to evaluate the work. 

A research proposal allows the sponsor to assess the sincerity of the researcher’s purpose, the clarity 
of his or her design, the extent of his or her relevant background material, and the researcher’s fitness 
for undertaking the project. Depending on the type of research and the sponsor, various aspects of a 
standard proposal design are emphasized. The proposal displays the researcher’s discipline, organiza- 
tion, and logic. It thus allows the research sponsor to assess both the researcher and the proposed design, 
to compare them against competing proposals on current organizational, scholastic, or scientific needs, 
and to make the best selection for the project. A poorly planned, poorly written, or poorly organized 
proposal damages the researcher’s reputation more than the decision not to submit a proposal. 

Comparison of the research project results with the proposal is also the first step in the process of evalu- 
ating the overall research. Comparing the final product with the stated objectives makes it is easy for the 
sponsor to decide if the research goal—a better decision on the management question—has been achieved. 

Another benefit of the proposal is the discipline it brings to the sponsor. Many managers, requesting 
research from an in-house departmental research project, do not adequately define the problem they 
are addressing. The research proposal acts as a catalyst for discussion between the person conducting 
the research and the manager. The researcher translates the management question, as described by the 
manager, into the research question and outlines the objectives of the study. Upon review, the manager 
may discover that the interpretation of the problem does not encompass all the original symptoms. The 
proposal, then, serves as the basis for additional discussion between the manager and the researcher 
until all aspects of the management question are understood. Parts of the management question may 
not be researchable, or at least not subject to empirical study. An alternate design, such as a qualitative 
or policy analysis study, may need to be proposed. Upon completion of the discussions, the sponsor 
and researcher should agree on a carefully worded research question. As Exhibit a-3 reveals, proposal 
development can work in an iterative fashion until the sponsor authorizes the research to proceed. 


>Exhibit a-3 Proposal Development 


Manager describes 
problem and states the 


management question. They 


disagree 


Researcher translates Researcher Manager and researcher 
management question explores discuss the proposal and 
into research question. alternative clarify and redefine the 
approaches. problem and objectives 
of the project. 


Researcher 
refines 
research 
question. 


Not 
accepted 


Accepted 


Researcher elaborates Researcher Manager Manager 
research question into prepares reviews approves 
investigative question(s). proposal. proposal. project. 
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Researcher Benefits 


A proposal is even more beneficial for the researcher than for the sponsor. The process of writing a 
proposal encourages the researcher to plan and review the project’s logical steps. Related manage- 
ment and research literature should be examined in developing the proposal. This review prompts the 
researcher to assess previous approaches to similar management questions and revise the research plan 
accordingly. Additionally, developing the proposal offers the opportunity to spot flaws in the logic, 
errors in assumptions, or even management questions that are not adequately addressed by the objec- 
tives and design. 

The in-house or contract researcher uses the approved research proposal as a guide throughout the 
investigation. Progress can be monitored and milestones noted. At completion, the proposal provides 
an outline for the final research report.* 

Like any other business, a contract researcher makes his or her profit from correctly estimating 
costs and pricing the research project appropriately. A thorough proposal process is likely to reveal 
all possible cost-related activities, thus making cost estimation more accurate. Because many of these 
cost-associated activities are related to time, a proposal benefits a researcher by forcing a time esti- 
mate for the project. These time and cost estimates encourage researchers to plan the project so that 
work progresses steadily toward the deadline. Since many people are inclined to procrastinate, having 
a schedule helps them work methodically toward the completion of the project. Researchers often 
develop Gantt charts of the logical research steps, similar to the one in Exhibit 5-11 in Chapter 5, as 
working documents when developing responses to RFPs. 


Types of Research Proposals 


In general, research proposals can be divided between those generated for internal and those gener- 
ated for external audiences. An internal proposal is done by staff specialists or by the research de- 
partment within the firm. External proposals sponsored by university grant committees, government 
agencies, government contractors, not-for-profit organizations, or corporations can be further classi- 
fied as either solicited or unsolicited. With few exceptions, the larger the project, the more complex 
the proposal. In public sector work, the complexity is generally greater than in a comparable private 
sector proposal. 

There are three general levels of complexity: exploratory studies, small-scale studies, and large- 
scale studies. These are noted in Exhibit a-4. The exploratory study generates the most simple research 
proposal. More complex and common in business is the small-scale study—either an internal study or 
an external contract research project. The large-scale professional study, worth up to several million 


>Exhibit a-4 Proposal Complexity 


Complexity 


Management proposals 


Internal Exploratory Small-scale} Large-scale 
study study study 


External Exploratory contract Small-scale Large-scale contract 
research contract research 


Government-sponsored research Large-scale 
contract research 


Academic proposals Research term paper Master’s thesis Doctoral thesis 
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>Exhibit a-5 Modules to Include in Proposals: A Comparison of 
Management-Oriented Proposals and Student Proposals 


Proposal Management Govern- milan 
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dollars, is the most complex proposal we deal with here. Government agency large-scale project RFPs 
usually generate proposals running several hundred pages and use the same modules that we discuss 
next. However, each agency has unique requirements, making generalized coverage beyond the scope 
of this text. 

Exhibit a-5 displays a set of modules for building a proposal. Their order can represent an outline 
for a proposal. Based on the type of proposal you are writing, you may choose the appropriate modules 
for inclusion. This is a general guide, and sometimes more or less than what is shown here is appropri- 
ate for a specific purpose. For example, most small-scale studies do not require a glossary of terms. 
Terms are defined within the body of the proposal. However, if the proposal deals with an esoteric 
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subject that is not familiar to management, it is appropriate to add a glossary. For a solicited study, the 
RFP will indicate both the content headings and their order. 


Internal Proposals 


Internal proposals are more succinct than external ones. At the least complex end of the continuum in 
Exhibit a-4, a one- to three-page memo from the researcher to management outlining the problem state- 
ment, study objectives, research design, and schedule is enough to start an exploratory study. Privately 
and publicly held businesses are concerned with how to solve a particular problem, make a decision, 
or improve an aspect of their business. Seldom do businesses begin research studies for other reasons. 
Regardless of the intended audience, in the small-scale proposal, the literature review and bibliography 
are consequently not stressed and can often be stated briefly in the research design. Since management 
insists on brevity, an executive summary is mandatory for all but the most simple of proposals (projects 
that can be proposed in a two-page memo do not need an executive summary). Schedules and budgets 
are necessary for funds to be committed. For the smaller-scale projects, descriptions are not required 
for facilities and special resources, nor is there a need for a glossary. Since managers familiar with the 
problem sponsor small projects, the associated jargon, requirements, and definitions should be included 
directly in the text. Also, the measuring instrument and project management modules are not required. 
Managers will typically leave this detail for researchers. 


External Proposals 


An external proposal is either solicited or unsolicited. A solicited proposal is often in response to an 
RFP. The proposal is likely competing against several others for a contract or grant. An unsolicited 
proposal represents a suggestion by a contract researcher for research that might be done. For an ex- 
ample, a consulting firm might propose a research project to a client that has retained the consultancy 
for other purposes. As another example, a research firm might propose an omnibus study to a trade 
association to address problems arising from a change in the cultural or political-legal environments. 
The unsolicited proposal has the advantage of not competing against others but the disadvantage of 
having to speculate on the ramifications of a management dilemma facing the firm’s management. In 
addition to being an outsider assessing an internal problem, the writer of an unsolicited proposal must 
decide to whom the document should be sent. Such proposals are often time-sensitive, so the window 
of opportunity might close before a redirected proposal finds its appropriate recipient. 

The most important sections of the external proposal are the objectives, design, qualifications, 
schedule, and budget. In contract research, the results and objectives sections are the standards 
against which the completed project is measured. The executive summary of an external proposal 
may be included within the letter of transmittal. As the complexity of the project increases, more 
information is required about project management and the facilities and special resources. As we 
move toward government-sponsored research, particular attention must be paid to each specification 
in the RFP. To ignore or not meet any specification is to automatically disqualify your proposal as 
“nonresponsive.’”? 

We offer a sample of an external proposal on the Online Learning Center. 


Structuring the Research Proposal 


Consider again Exhibit a-5. Using this reference, you can put together a set of modules that tailors your 
proposal to the intended audience. Each of the following modules is flexible, so its content and length 
may be adapted to specific needs. 


Executive Summary 


The executive summary allows a busy manager or sponsor to understand quickly the thrust of the 
proposal. It is essentially an informative abstract, giving executives the chance to grasp the essentials 
of the proposal without having to read the details.° The goal of the summary is to secure a positive 
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evaluation by the executive who will pass the proposal on to the staff for a full evaluation. As such, 
the executive summary should include brief statements of the management dilemma and management 
question, the research objectives/research question(s), and the benefits of your approach. If the pro- 
posal is unsolicited, a brief description of your qualifications is also appropriate. 


Problem Statement 


This section needs to convince the sponsor to continue reading the proposal. You should capture the 
reader’s attention by stating the management dilemma, its background, its consequences, and the re- 
sulting management question. The importance of answering the management question should be em- 
phasized here if a separate module on the importance/benefits of the study is not included later in the 
proposal. In addition, this section should include any restrictions or areas of the management question 
that will not be addressed. 

Problem statements too broadly defined cannot be addressed adequately in one study. It is important 
that the management question distinguish the primary problem from related problems clearly. Be sure 
your problem statement is clear without the use of idioms or clichés. After reading this section, the 
potential sponsor should know the management dilemma and the question, its significance, and why 
something should be done to change the status quo.’ 


Research Objectives 


This module addresses the purpose of the investigation. It is here that you lay out exactly what is being 
planned by the proposed research. In a descriptive study, the objectives can be stated as the research 
question. Recall that the research question can be further broken down into investigative questions. If 
the proposal is for a causal study, then the objectives can be restated as a hypothesis. 

The objectives module flows naturally from the problem statement, giving the sponsor specific, 
concrete, and achievable goals. It is best to list the objectives either in order of importance or in general 
terms first, and then moving to specific terms (i.e., research question followed by underlying investiga- 
tive questions). The research question(s) (or hypotheses, if appropriate) should be separated from the 
flow of the text for quick identification. 

The research objectives section is the basis for judging the remainder of the proposal and, ulti- 
mately, the final report. Verify the consistency of the proposal by checking to see that each objective is 
discussed in the research design, data analysis, and results sections. 


Literature Review 


The literature review section examines recent (or historically significant) research studies, company 
data, or industry reports that act as a basis for the proposed study. Begin your discussion of the related 
literature and relevant secondary data from a comprehensive perspective, moving to more specific 
studies that are associated with your problem. If the problem has a historical background, begin with 
the earliest references. A literature review might reveal that the sponsor can answer the management 
question with a secondary data search rather than the collection of primary data. 

Avoid the extraneous details of the literature; do a brief review of the information, not a compre- 
hensive report. Always refer to the original source. If you find something of interest in a quotation, 
find the original publication and ensure you understand it. In this way, you will avoid any errors of 
interpretation or transcription. Emphasize the important results and conclusions of other studies, the 
relevant data and trends from previous research, and particular methods or designs that could be 
duplicated or should be avoided. Discuss how the literature applies to the study you are proposing; 
show the weaknesses or faults in the design, discussing how you would avoid similar problems. If 
your proposal deals solely with secondary data, discuss the relevance of the data and the bias or lack 
of bias inherent in it. 

The literature review may also explain the need for the proposed work to appraise the shortcom- 
ings and/or informational gaps in secondary data sources. This analysis may go beyond scrutinizing 
the availability or conclusions of past studies and their data, to examining the accuracy of secondary 
sources, the credibility of these sources, and the appropriateness of earlier studies. 
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Close the literature review section by summarizing the important aspects of the literature and inter- 
preting them in terms of your problem. Refine the problem as necessary in light of your findings. 


Importance/Benefits of the Study 


In this section you describe explicit benefits that will accrue from your study. The importance of “doing 
the study now” should be emphasized. Usually, this section is not more than a few paragraphs. If you 
find it difficult to write, then you have probably not adequately clarified the management dilemma. 
Return to the analysis of the problem and ensure, through additional discussions with your sponsor or 
your research team or by a reexamination of the literature, that you have captured the essence of the 
problem. 

This section also requires you to understand what is most troubling to your sponsor. If it is a 
potential union activity, you cannot promise that an employee survey will prevent unionization. 
You can, however, show the importance of this information and its implications. This benefit may 
allow management to respond to employee concerns and forge a linkage between those concerns and 
unionization. 

The importance/benefits section is particularly important to the unsolicited external proposal. You 
must convince the sponsoring organization that your plan will meet its needs. 


Research Design 


Up to now, you have told the sponsor what the problem is, what your study goals are, and why it is 
important for you to do the study. The proposal has presented the study’s value and benefits. The design 
module describes what you are going to do in technical terms. This section should include as many 
subsections as needed to show the phases of the project. Provide information on your proposed design 
for tasks such as sample selection and size, data collection method, instrumentation, procedures, and 
ethical requirements. When more than one way exists to approach the design, discuss the methods you 
have rejected and why your selected approach is superior. 


Data Analysis 


A brief section on the methods used for analyzing the data is appropriate for large-scale contract 
research projects and doctoral theses. With smaller projects, the proposed data analysis would be 
included within the research design section. It is in this section that you describe your proposed 
handling of the data and the theoretical basis for using the selected techniques. The object of this 
section is to assure the sponsor you are following correct assumptions and using theoretically sound 
data analysis procedures. 

This module is often an arduous section to write. You can make it easier to write, read, and under- 
stand your data analysis by using sample charts and tables featuring “dummy” data. 

The data analysis section is so important to evaluating contract research proposals that the researcher 
should contact an expert to review the latest techniques available for use in the particular research study 
and compare these to the proposed techniques. When there is no statistical or analytical expertise in the 
company, sponsors are more likely to hire professional help to interpret the soundness of this section. 


Nature and Form of Results 


Upon finishing this section, the sponsor should be able to go back to the statement of the management 
question and research objectives and discover that each goal of the study has been covered. One should 
also specify the types of data to be obtained and the interpretations that will be made in the analysis. If 
the data are to be turned over to the sponsor for proprietary reasons, make sure this is reflected. Alter- 
natively, if the report will go to more than one sponsor, that should be noted. 

This section also contains the contractual statement telling the sponsor exactly what types of in- 
formation will be received. Statistical conclusions, applied findings, recommendations, action plans, 
models, strategic plans, and so forth, are examples of the forms of results. 
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Qualifications of Researchers 


This section should begin with the principal investigator and then provide similar information on all 
individuals involved with the project. Two elements are critical: 


1. Professional research competence (relevant research experience, the highest academic degree 
held, and memberships in business and technical societies). 


2. Relevant management experience.® 


With so many individuals, research specialty firms, and general consultancies providing research ser- 
vices, the sponsor needs assurance that the researcher is professionally competent. Past research ex- 
perience is the best barometer of competence, followed by the highest academic degree earned. To 
document relevant research experience, the researcher provides concise descriptions of similar proj- 
ects. Highest degree usually follows the person’s name (e.g., S. Researcher, PhD in Statistics). Society 
memberships provide some evidence that the researcher is cognizant of the latest methodologies and 
techniques. These follow the relevant research experience as a string or bulleted list, with organization 
name followed by term of membership and any relevant leadership positions. 

Researchers are increasingly in the business of providing advice, not just research services. And 
businesses are looking for quality advice. Comparatively, the researcher who demonstrates relevant 
management or industry experience will be more likely to receive a favorable nod to his or her pro- 
posal. The format of this information should follow that used for relevant research experience. The 
entire curriculum vitae of each researcher need not be included unless required by the RFP. However, 
researchers often place complete vitae information in an appendix for review by interested sponsors. 

Research companies often subcontract specific research activities to firms or individuals that spe- 
cialize or offer specific resources or facilities. This is especially true for studies involving qualitative 
research techniques such as in-depth personal interviews and focus groups. Usually brief profiles of 
these companies are provided in this section only if their inclusion enhances the credibility of the 
researcher. Otherwise, profiles of such subcontractors are included in an appendix of the final report, 
rather than in the proposal. 


Budget 


The budget should be presented in the form the sponsor requests. For example, some organizations 
require secretarial assistance to be individually budgeted, whereas others insist it be included in the 
research director’s fees or the overhead of the operation. In addition, limitations on travel, per diem 
rates, and capital equipment purchases can change the way in which you prepare a budget. 

Typically, the budget should be no more than one to two pages. Exhibit a-6 shows one format that 
can be used for small contract research projects. Additional information, backup details, quotes from 
vendors, and hourly time and payment calculations should be put into an appendix if required or kept 
in the researcher’s file for future reference. 

The budget statement in an internal research proposal is based on employee and overhead costs. The 
budget presented by an external research organization is not just the wages or salaries of its employees 
but the person-hour price that the contracting firm charges. 

The detail the researcher presents may vary depending on both the sponsors’ requirements and 
the contracting research company’s policy. Some research companies, particularly in database and 
computerized analysis areas, quote on the basis of “man-machine hours” involved in a project. The 
man-machine hour is the hourly fee charged for a person with computer hardware and organizational 
resources. Here, rather than separating the “other costs,” Exhibit a-6 shows these costs embedded in 
a combined rate. One reason why external research agencies avoid giving detailed budgets is the pos- 
sibility that disclosures of their costing practices will make their calculations public knowledge, reduc- 
ing their negotiating flexibility. Since budget statements embody a work strategy depicted in financial 
terms that could be used by the recipient of the proposal to develop a replicate research plan, vendors 
are often doubly careful. 

The budget section of an external research contractor’s proposal states the total fee payable for 
the assignment. When it is accompanied by a proposed schedule of payment, this is frequently 
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>Exhibit a-6 Sample Proposal Budget for a Research Program 


Budget Items Rate Total Days Charge 
A. Salaries 
1. Research director, Jason Henry $200/hr. 20 hours $ 4,000 
2. Associate 100/hr. 10 hours 1,000 
8. Research assistants (2) 20/hr. 300 hours 6,000 
4. Secretarial (1) 12/h. 100 hours 1,200 
Subtotal $12,200 


B. Other costs 
5. Employee services and benefits 


6. Travel $ 2,500 
7. Office supplies 100 
8. Telephone 800 
9. Rent 

10. Other equipment 
11. Publication and storage costs 100 
Subtotal $ 3,500 
C. Total of direct costs $15,700 
D. Overhead support $ 5,480 
E. Total funding requested $21,180 


detailed in a purchase order. Like other large-ticket-price services delivered over time in stages (e.g., 
building a home), payments can be paid at stages of completion. Sometimes a retainer is paid at the 
beginning of the contract, then a percentage at an intermediate stage, and the balance on completion 
of the project. 

It is extremely important that you retain all information you use to generate your budget. If you use 
quotes from external contractors, get the quotation in writing for your file. If you estimate time for in- 
terviews, keep explicit notes on how you made the estimate. When the time comes to do the work, you 
should know exactly how much money is budgeted for each particular task.’ 

Some costs are more elusive than others. Do not forget to build the cost of proposal writing into 
your fee. Publication and delivery of final reports can be a last-minute expense that may be easily over- 
looked in preliminary budgets. 


Schedule 


Your schedule should include the major phases of the project, their timetables, and the milestones that 
signify completion of a phase. For example, major phases may be (1) exploratory interviews, (2) final 
research proposal, (3) questionnaire revision, (4) field interviews, (5) editing and coding, (6) data 
analysis, and (7) report generation. Each of these phases should have an estimated time schedule and 
people assigned to the work. 

It may be helpful to you and your sponsor if you chart your schedule. You can use a Gantt chart, 
shown in Chapter 5, Exhibit 5-11. Alternatively, if the project is large and complex, a critical path 
method (CPM) of scheduling may be included.'® In a CPM chart, the nodes represent major mile- 
stones, and the arrows suggest the work needed to get to the milestone. More than one arrow pointing to 
a node indicates all those tasks must be completed before the milestone has been met. Usually a number 
is placed along the arrow showing the number of days or weeks required for that task to be completed. 
The pathway from start to end that takes the longest time to complete is called the critical path, because 
any delay in an activity along that path will delay the end of the entire project. An example of a CPM 
chart is shown in Exhibit a-7. Software programs designed for project management simplify scheduling 
and charting the schedule. Most are available for personal computers. 
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>Exhibit a-7 CPM Schedule 


Client 


Final check of background records 


Conduct 
exploratory 


interviews interviews 


Literature Develop 


review 


Complete Field Code data 
questionnaire interviewing _ and analyze 


Meet 
. client 


Complete executive 


3 : : Interview 
questionnaire 


proposal , 
executives 


Arrange executive 
appointments 


Milestones: Critical Path: 


S-1-3-4-7-8-9-E 


3 Proposal approval 
7 Interviews completed Time to Completion: 
9 Final report completed 40 working days 


Facilities and Special Resources 


Often, projects will require special facilities or resources that should be described in detail. For ex- 
ample, a contract exploratory study may need specialized facilities for focus group sessions. Computer- 
assisted telephone or other interviewing facilities may be required. Alternatively, your proposed data 
analysis may require sophisticated computer algorithms, and therefore you need access to an adequate 
system. These requirements will vary from study to study. The proposal should carefully list the rel- 
evant facilities and resources that will be used. The costs for such facility use should be detailed in 
your budget. 


Project Management 


The purpose of the project management section is to show the sponsor that the research team is orga- 
nized in a way to do the project efficiently. A master plan is required for complex projects to show how 
all the phases will be brought together. The plan includes: 

e The research team’s organization. 

e Management procedures and controls for executing the research plan. 

e Examples of management and technical reports. 

e The research team’s relationship with the sponsor. 

e Financial and legal responsibility. 

e Management competence. 

Tables and charts are most helpful in presenting the master plan. The relationships between 
researchers and assistants need to be shown when several researchers are part of the team. Sponsors 
must know that the director is an individual capable of leading the team and acting as a useful liai- 


son to the sponsor. In addition, procedures for information processing, record control, and expense 
control are critical to large operations and should be shown as part of the management procedures. 
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The type and frequency of progress reports should be recorded so that the sponsor can expect to be 
kept up to date and the researchers can expect to be left alone to do research. The sponsor’s limits on 
control during the process should be delineated. 

This section also discusses any details such as printing facilities, clerical help, or information pro- 
cessing capabilities to be provided by the sponsor rather than the researcher. In addition, rights to the 
data, the results, and authority to speak for the researcher and for the sponsor are included. 

Payment frequency and timing are also covered in the master plan. Finally, proof of financial 
responsibility and overall management competence is provided. 


Bibliography 


For all projects that require a literature review, a bibliography is necessary. Use the bibliographic 
format required by the sponsor. If none is specified, a standard style manual will provide the details 
necessary to prepare the bibliography.'' Many of these sources also offer suggestions for successful 
proposal writing. 


Appendices 


Glossary The researcher should include a glossary of terms whenever there are many words 
unique to the research topic and not understood by the general management community. This is a 
simple section consisting of terms and definitions, similar in format to the glossary in this textbook. 
Also, the researcher should define any acronyms used, even if they are defined within the text (e.g., 
CATI for “computer-assisted telephone interviewing”). 


Measurement Instrument For large projects, it is appropriate to include samples of the measure- 
ment instruments if they are available when you assemble the proposal. This allows the sponsor to 
discuss particular changes in one or more of the instruments. If the proposal includes the development 
of a custom-designed measurement instrument, omit this appendix section. 


Other Any detail that reinforces the body of the proposal can be included in an appendix. This 
includes researcher vitae, profiles of firms or individuals to which work will be subcontracted, budget 
details, and lengthy descriptions of special facilities or resources. 

To see how some of these elements were incorporated in the MindWriter research proposal, see 
Exhibit a-8. 


Evaluating the Research Proposal 


Proposals are subject to either formal or informal reviews. Formal reviews are regularly done for solic- 
ited proposals. The formal review process varies but typically includes: 


¢ Development of review criteria, using RFP guidelines. 
e Assignment of points to each criterion, using a universal scale. 
e Assignment of a weight for each criterion, based on importance of each criterion. 


¢ Generation of a score for each proposal, representing the sum of all weighted criterion scores. 


The sponsor should assign the criteria, the weights, and the scale to be used for scoring each criterion 
before the proposals are received. The proposal then should be evaluated with this checklist of criteria 
in hand. Points are recorded for each criterion reflecting the sponsor’s assessment of how well the pro- 
posal meets the company’s needs relative to that criterion (e.g., 1 through 10, with 10 being the largest 
number of points assigned to the best proposal for a particular criterion). After the review, the weighted 
criterion scores are added to provide a cumulative total. The proposal with the highest number of points 
wins the contract. 
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>Exhibit a-8 Proposal for MindWriter CompleteCare Satisfaction Research 


In preparing a proposal for Gracie Uhura, product manager at MindWriter Corporation, Jason and Sara decided to exclude the “execu- 
tive summary” for two reasons: The proposal is short and the essentials will be contained in the cover letter. The proposal follows the 
components discussed in this chapter. It is an appropriate adaptation for an internal, small-scale study. The module “qualification of 
researcher” was not needed because MindWriter’s employee solicited the proposal, thus had prejudged the researcher’s qualifications. 


Repair Process Satisfaction Proposal 
MindWriter Corporation CompleteCare Program 


Problem Statement 


MindWriter Corporation has recently created a service and repair program, CompleteCare, for its portable/laptop/notebook computers. 
This program promises to provide a rapid response to customers’ service problems. 

MindWriter is currently experiencing a shortage of trained technical operators in its telephone center. The package courier, 
contracted to pick up and deliver customers’ machines to CompleteCare, has provided irregular execution. MindWriter has also 
experienced parts availability problems for some machine types. 

Recent phone logs at the call center show complaints about CompleteCare; it is unknown how representative these complaints are 
and what implications they may have for satisfaction with MindWriter products. 

Management desires information on the program’s effectiveness and its impact on customer satisfaction to determine what should 
be done to improve the CompleteCare program for MindWriter product repair and servicing. 


Research Objectives 


The purpose of this research is to discover the level of satisfaction with the CompleteCare service program. Specifically, we intend to 
identify the component and overall levels of satisfaction with CompleteCare. Components of the repair process are important targets 
for investigation because they reveal: 

(1) How customer tolerance levels for repair performance affect overall satisfaction, and 

(2) Which process components should be immediately improved to elevate the overall satisfaction of those MindWriter customers 
experiencing product failures. 

We will also discover the importance of types of product failure on customer satisfaction levels. 


Importance/Benefits 


High levels of user satisfaction translate into positive word-of-mouth product endorsements. These endorsements influence the 
purchase outcomes for (1) friends and relatives and (2) business associates. 

Critical incidents, such as product failures, have the potential to either undermine existing satisfaction levels or preserve and 
even increase the resulting levels of product satisfaction. The outcome of the episode depends on the quality of the manufacturer’s 
response. 

An extraordinary response by the manufacturer to such incidents will preserve and enhance user satisfaction levels to the point 
that direct and indirect benefits derived from such programs will justify their costs. 

This research has the potential for connecting to ongoing MindWriter customer satisfaction programs and measuring the long-term 
effects of CompleteCare (and product failure incidents) on customer satisfaction. 


Research Design 


Exploration: Qualitative. We will augment our knowledge of CompleteCare by interviewing the service manager, the call center 
manager, and the independent package company’s account executive. Based on a thorough inventory of CompleteCare’s internal 
external processes, we propose to develop an online survey. 

Questionnaire Design. An email-recruited, online, self-administered questionnaire offers the most cost-effective method for securing 
feedback on the effectiveness of CompleteCare. The introduction on the questionnaire will be a variation of MindWriter’s current 
advertising campaign. 

Some questions for this instrument will be based on the investigative questions we presented to you previously, and others will be 
drawn from the executive interviews. We anticipate a maximum of 12 questions. A new five-point expectation scale, compatible with 
your existing customer satisfaction scales, is being designed. 

We understand that you and Mr. Malraison are interested in verbatim comments. A comments/suggestions question will be included. 
In addition, we will work out a code block that captures the call center’s reference number, model, and item(s) serviced. 
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>Exhibit a-8 Proposal for MindWriter CompleteCare Satisfaction Research (concluded) 


Logistics. We recommend using SurveyMonkey as the distributor of the questionnaire, for its ease of use and timeliness of data 
collection. 

Pilot Test. We will test the questionnaire with a small sample of customers using your tech-line operators. This will contain your 
costs. We will then revise the questions and forward them to our graphics designer for layout. The instrument will then be submitted to 
you for final approval. 

Evaluation of Nonresponse Bias. A random sample of 100 names will be secured from the list of customers who choose not to 
participate in the online survey. CompleteCare records will be used for establishing the sampling frame. Nonresponders will be 
interviewed on the telephone and their responses compared statistically to those of the responders. 


Data Analysis 


We will review and send you a weekly report listing customers who are dissatisfied (score a “1” or “2”) with any item of the 
questionnaire or who submit a negative comment. This will improve your timeliness in resolving customer complaints. Each month, 
we will provide you with a report consisting of frequencies and category percentages for each question. Visual displays of the data 
will be in bar chart/histogram form. We propose to include at least one question dealing with overall satisfaction (with CompleteCare 
and/or MindWriter). This overall question would be regressed on the individual items to determine each item’s importance. 
A performance grid will identify items needing improvement with an evaluation of priority. Other analyses can be prepared on a time 
and materials basis. 

The open-ended questions will be summarized and reported by model code. If you wish, we also can provide content analysis for 
these questions. 


Results: Deliverables 


1. Development and production of an online questionnaire. 

2. Weekly exception reports (transmitted electronically) listing customers who meet the dissatisfied customer criteria. 
3. Monthly reports as described in the data analysis section. 

4. Upload of each month's data to a secure FTP site, by the fifth working day of each month. 


Budget 


Questionnaire Design. Development of a customized scale and online survey, incorporating the MindWriter logo using a customized 
SurveyMonkey account. 


Cost Summary 


Interviews $ 1,550.00 
Travel costs 2,500.00 
Questionnaire development 1,850.00 
Equipment/supplies 1,325.00 
Graphics design 800.00 
Survey Monkey secure account 200.00 
Secure FTP site 200.00 
Monthly data files (each) 50.00 
Monthly reports (each) 1,850.00 

Total start-up costs $10,325.00 


Monthly run costs $ 1,030.00 
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Several people, each of whom may be assigned to a particular section, typically review long and 
complex proposals. The formal method is most likely to be used for competitive government, univer- 
sity, or public sector grants and also for large-scale contracts. 

Small-scale contracts are more prone to informal evaluation. In an informal review, the project 
needs, and thus the criteria, are well understood but are not usually well documented. In contrast to the 
formal method, a system of points is not used and the criteria are not ranked. The process is more quali- 
tative and impressionistic. Exhibit a-9 shows Sara Arens’s informal review of a proposal presented to 
her as a member of her county’s Economic Development Council. 

In practice, many factors contribute to a proposal’s acceptance and funding. Primarily, the content 
discussed above must be included to the level of detail required by the sponsor’s RFP. Beyond the 
required modules, other factors can quickly eliminate a proposal from consideration or improve the 
sponsor’s reception of the proposal, among them: 


e Neatness. 

° Organization in terms of being both logical and easily understood. 

e Completeness in fulfilling the RFP’s specifications, including budget and schedule. 
e Appropriateness of writing style. 


e Submission within the RFP’s timeline. 


Although a proposal produced on a word processor and bound with an expensive cover will not 
overcome design or analysis deficiencies, a poorly presented, unclear, or disorganized proposal will 
not get serious attention from the reviewing sponsor. Given that multiple reviewers may be evaluat- 
ing only a given section, the reviewer should be able to page through the proposal to any section of 
interest. 

In terms of the technical writing style of the proposal, the sponsor must be able to understand the 
problem statement, the research design, and the methodology. The sponsor should clearly understand 
why the proposed research should be funded and the exact goals and concrete results that will come 
from the study. 

The proposal also must meet specific RFP guidelines set by the sponsoring company or agency, 
including budgetary restrictions and schedule deadlines. A schedule that does not meet the expected 
deadlines will disqualify the proposal. A budget that is too high for the allocated funds will be rejected. 
Conversely, a low budget compared to competing proposals suggests that something is missing or there 
is something wrong with the researchers. 

Finally, a late proposal will not be reviewed. Although current project disqualification due to late- 
ness may appear to be the worst result here, there is a possible longer-term effect created. Lateness 
communicates a level of disrespect for the sponsor—that the researcher’s schedule is more important 
than the sponsor’s. A late proposal also communicates a weakness in project management, which 
raises an issue of professional competence. This concern about competence may continue to plague the 
researcher during future project proposal reviews. 
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>Exhibit a-9 Informal Proposal Review 


Sara Arens 


SSS 
200 ShellPoint Tower 
Palm Beach, Florida 33480 
Mr. Harry Shipley, President 
Economic Development Council 
1800 ShellPoint Tower 
Palm Beach, Florida 33480 


Dear Harry, 


I have reviewed Robert Buffet’s proposal for an investigation of the job creation practices of local companies and, in short, I am 
very much concerned with several aspects of the “proposal.” It is not really a proposal at all, because it lacks sufficient detail. 

First, let me mention that I shared Buffet’s proposal with Mr. Jason Henry, my partner. Mr. Buffet and his organization may one 
day represent competition for us, and you must therefore be aware of a potential conflict of interest and perhaps discount the opinions 
stated here. Since I am delivering this letter to you in two days rather than the two weeks you requested, you may wish to discuss my 
comments with others. 

What you and Mr. Buffet gave me is an abbreviated research plan for our county, but since it lacks many features found in a com- 
prehensive proposal, I immediately saw it was not the full proposal that had been funded by the state commerce secretary. I called 
Tallahassee and reached a young woman who hemmed and hawed and refused to say if she was authorized to mail me the full pro- 
posal. Finally, I gave up arguing and gave her your address and told her she could mail it to you. 

I then made several calls to people in Tallahassee whom I know. Did you know that this research idea is being floated by our 
senior U.S. senator, who is eager to throw a monkey wrench into the president’s tax incentives plan? The senator whispered it to the 
governor and the governor whispered it to her commerce secretary, and here we are. 

The problem statement is rather long and convoluted, but, in short, it poses the questions, Are new high-tech companies creating 
jobs for residents of our county? Or are they bringing technical and manufacturing workers from outside the state and bypassing the 
local work force? Or are they doing research in these companies with a low level of manufacturing job creation? Or are they investing 
in ‘smart’ capital equipment that does not create jobs? If you cut through the verbiage, I think you can see the project is dead on the 
mark with its questions. 

The research objectives section is fairly straightforward. Buffet’s people are going to identify all the companies in this county in 
the NAICS code groups associated with “high tech” and collect information on the number of locally hired employees in various job 
categories, chiefly in production, and also collect data on capital investments, debt, and other financial data, which Jason says makes 
good sense to collect and ought to be easy to do. 

There is a section called Importance of the Study, which is full of platitudes and does not get around to mentioning the pending tax 
legislation. But at least the platitudes are brief. 

I become nervous in the Design section. It calls for Mr. Buffet’s group to go on site with a “team” and conduct in-depth interviews with 
the chief operating officer (COO), treasurer, and comptroller of each company and enter the data into a spreadsheet. I have double-checked 
this with Jason and also with a banker friend, and both of them assure me that a simple questionnaire might be mailed to the COO. There 
is no need whatsoever to send in a team to conduct open-ended interviews. Although there might be a noncompliance problem associated 
with filling out a form, this might appropriately be attended to by pointing out the auspices—the state commerce secretary and your Eco- 
nomic Development Council—with an interview request as a last resort. 

The proposal contains no budget and no specific list of researchers who will comprise the team. The firm would have carte blanche 
to go in with anyone on their payroll and try to induce the subjects to stray beyond the stated research objectives to talk about anything 
at all. Obviously such license would be a marketing tool and might allow the researchers to collect a list of researchable problems not 
related to the secretary’s needs, as stated in the problem section. 

I strongly advise you to tell Mr. Buffet to collect the information through a simple mail survey. Offer to send it out under your 
council’s letterhead, or see if you can get the commerce office or even the governor’s office to send it out. But do not subject your 
local business community to unstructured, free-ranging visits, which are clearly not justified by the research objectives. 


Sincerely, 


Se ara 
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Covering Kids RFP 


Wirthlin Worldwide earned the Ogilvy Research Award for creative and effective research instrumental in the 
development of the Covering Kids advertising campaign. This RFP from sponsor Robert Wood Johnson Founda- 
tion started the process that resulted in enrolling more than one million additional children for a health insurance 
initiative. 


March 13, 20xx 


Name 

Firm 

Address 

City, State Zip code 


Dear XX XX: 


As you know, we are working with GMMB&A to support the national Covering Kids Initiative (CKI). We appre- 
ciate your recent response to a proposal to support this effort’s marketing research requirements. Since that time, 
we have further refined our requirements. We hope that you will be willing to review this request, and revise your 
previous proposal in any ways needed to meet these altered needs. 

The Covering Kids Initiative is a $47 million national program of the Foundation that works to enroll eligible 
children in Medicaid and the Federal-state Children’s Health Insurance Programs (SCHIP). Three-year grants for 
the Covering Kids Initiative support coalitions in 49 states and the District of Columbia. These coalitions conduct 
outreach initiatives and work to simplify and coordinate the enrollment processes for health coverage programs for 
low-income children. In its first two years of activity, the CKI has focused largely on simplifying the enrollment pro- 
cess. During the second year, in addition to continuing a focus on simplification and coordination, there will be target 
marketing campaigns to encourage adults to enroll eligible children in both the SCHIP and Medicaid programs. 

The Foundation will work with its Covering Kids communications contractor to support these CKI coalitions in 
marketing, advertising, public relations, coalition building, and cause-related partnerships at the national and state 
levels. The tasks described here will help provide direction for the strategic development of communications and pro- 
vide support for testing and measuring communications campaigns in six markets prior to introduction nationwide. 


Background 


There are approximately five million uninsured children in the US eligible for either SCHIP or Medicaid. While 
income eligibility requirements in the federally funded SCHIP programs vary from state-to-state, they all gen- 
erally cover children in households of four with incomes up to $33,400 (higher in some states). About half of 
the eligible-uninsured Americans are non-Hispanic white, about 30% are African-American, another 20% are 
Hispanic/Latino. Although the numbers are much smaller, a large proportion of Native-Americans are also eligible 
but not covered. 

There are many reasons why so many eligible children are not enrolled. Some primary barriers to enrollment 
are: lack of awareness of the availability of health programs, especially SCHIP; lack of knowledge of eligibility 
criteria for these programs; complicated/onerous application processes; a stigma attached to government-funded 
health care programs (especially for working parents); the lack of outreach experience and expertise (most states 
have never conducted outreach for programs like Medicaid). 
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The primary challenge for this project is to create a nationwide campaign to enroll children—yet the “fulfillment 
mechanisms” (the state SCHIP and Medicaid programs) vary from state to state. Many states have developed their 
own distinct marketing and branding campaigns, so that the SCHIP programs in Connecticut (HUSKY B) and in 
Georgia (PeachCare) and in Illinois (KidsCare) resemble traditional private health plans more than they do govern- 
ment programs based on income eligibility. A national 1-877-KIDS-NOW phone number is in use that seamlessly 
routes calls through to the appropriate state program office. We will likely use that toll-free number as a marketing 
and fulfillment mechanism for this effort. 

The communications campaign will target specific groups of parents and other adults who could play a key role 
in enrolling eligible children in existing programs. Specific messages will be tested for use with subsets of low 
income Americans, including African-Americans, Hispanic/Latino, Native Americans and others. The Campaigns 
will first be tested and measured in six regional markets before national advertising begins. Ad buys and other 
communications activities will be coupled with local enrollment events. Communications activities and enrollment 
events will likely intensify twice annually, during the back-to-school and winter cold and flu seasons. 


Contractual Needs 


Requirements for market research and evaluation support are described in the two tasks below. 


Task 1. MARKET RESEARCH. Design, conduct, analyze, and provide conclusions relevant to communications 
planning. 


Task 1a. Develop an in-depth comprehensive profile (through a series of in-depth interviews) of the families 
of eligible-but-uninsured children—who are they, where are they, why are they not enrolled, the most effective 
messages/concepts to move individuals in specific groups to enroll in SCHIP/Medicaid, what messages/words/ 
concepts are definite turn-offs among specific groups, etc. 

Of particular interest: 


e Hispanic/Latino rural/urban 


African-American rural/urban 


Native-American rural/urban 
White rural/urban 
Parents of children enrolled in SCHIP 


Parents of children enrolled in Medicaid 


Parents of uninsured children eligible for SCHIP and/or Medicaid who haven’t applied 


Parents of uninsured children eligible for SCHIP and/or Medicaid who have applied but are not enrolled 
Level of effort: approximately 120 in-depth interviews. [Alternative suggestions invited. ] 


Task 1b. Qualitative research among opinion leaders: their perceptions of SCHIP; their definition of success/ 
failure, etc. These might include federal and state legislative staff, regulatory staff, child health advocates, constitu- 
ency group leaders, and media gatekeepers. 

Level of effort: approximately 25 in-depth interviews. [Alternative suggestions invited. ] 


Evaluation Task 


Task 1. A comprehensive national survey. This survey will: help direct communications development; provide con- 
tent for news placement; provide a pre-campaign benchmark (baseline data). We anticipate repeating this survey in the 
future to help gauge change and progress. However, at this time we are interested only in one benchmark survey. 

We are considering two options for survey sampling: a) a national sample including oversamples of lower- 
income families as described above, or b) a sample consisting of lower-income families with sufficient subsets 
(described above) to be statistically reliable. We are interested in receiving recommendations regarding which op- 
tion to pursue as well as a description of how this work would be done. 


Task 2. An evaluation of the media campaign in 6 test markets. Because the national advertising and public rela- 
tions components of the communication campaign will be large in scope and level of effort, this test market phase 
will be used to test and refine media messages, techniques, and decisions. The test market evaluation is critical to 
decision making for this effort. 
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Six mid-sized media markets will be selected to obtain a mix of the targeted demographic groups and geographic 
diversity. The Foundation will provide the list of selected sites to the contractor. The advertising and public rela- 
tions test phase will span 4-6 weeks, planned for late-August—early-September 2000. The market test will be 
planned and executed through close collaboration with the communications contractor, the Covering Kids National 
Program Office and coalitions in the target markets, and the Foundation. It will be designed to gauge heightened 
awareness, perceptions of target audiences, willingness to apply, and impact of campaign on overcoming any at- 
titudinal barriers to applying. 

This market test will include 


Task 2a. Benchmark survey—a random sample telephone survey with oversamples of target audiences. To 
include both benchmarking questions (such as awareness, attitude, intention measures) and message development 
questions (such as questions about message concepts, language). 


Task 2b. Post-campaign survey—a brief telephone follow up survey using same sampling; to include questions 
to assess recall/awareness, attitudes, intentions. 


Task 2c. Tracking of callers to the promoted toll-free number—the Foundation/National Program Office will 
provide a liaison to the toll-free manager(s). At this time it is not clear whether this will include only the national 
toll-free number or/and some state operated numbers. This task will include both compiling and analyzing call data 
and identifying ways to re-contact callers to assess further actions. 


Task 2d. Follow-up with callers—brief telephone survey to identify any questions taken. Phone numbers will be 
provided through the liaison described above. [Alternative method of assessment may be needed. ] 


Alternative suggestions for test market evaluation, with approximately the same level of effort required, are 
acceptable. 


Anticipated Time Schedule 


April 20xx contract awarded 
begin development of all tasks 


May 20xx conduct national survey 


June 20xx conduct research with potential beneficiaries, opinion leaders 
conduct benchmark survey in 6 test markets 


August 20xx begin ads in test markets 
begin telephone tracking 


September 20xx conduct testing market post-test surveys 
begin follow-up with telephone callers 


October 20xx present findings of national survey 
present findings from caller callbacks 


Proposal Instructions 


We invite you to submit a proposal addressing one or both of the tasks described above. Your proposal should address: 


° your approach to conducting the work 


° any alternative methods or procedures you would like to suggest for accomplishing the work described 
(optional) 


° a discussion of any anticipated challenges to completing these tasks, and how you would propose handling 
the challenges 


* comments on methodology and other recommendations for producing the needed information 


° specific work to be performed, description of the deliverables to be provided, and all costs (included out-of- 
pocket costs), by task 


e relevant experience and expertise 


° references 
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The Foundation is not seeking lengthy or elaborate proposals. Rather proposals should succinctly provide informa- 
tion that will permit a review using the criteria listed below. 


Review Criteria 


In reviewing proposals, we will consider: 


* your approach to the needs and tasks described here, including recommended methodology 


anticipated problems and how these would be handled 


company and staff experience in conducting similar market research and evaluation 


company and staff experience in conducting qualitative and quantitative research with a similar population 


company and staff experience with a health insurance or similar health care issue 


personnel, task and time line, project management 


proposed budget 


ability to respond to time schedule. 


In addition we will expect that your company has no conflicts of interest with the Foundation. 
Proposals will be due no later than COB Monday, March 27, delivered to 


Stuart Schear (Four Copies) Kristine Hartvigsen (Three Copies) 
Senior Communications Officer Covering Kids National Program Office 
The Robert Wood Johnson Foundation Address 

Address City, State Zip code 

City, State Zip code Phone 

Phone 

David Smith (Four Copies) Elaine Bratic Arkin (One Copy) 
GMMB&A Address 

Address City, State Zip code 

City, State Zip code Phone 

Phone 


We anticipate notifying those who submit proposals of our decision no later than April 1, 2000. I am available by 
e-mail @ smr@rwif.org to answer any questions you might have. The following websites offer a wealth of infor- 
mation as well: 


<http://www.coveringkids.org>—the website for “Covering Kids,” the RWJF-funded initiative that these 
tasks will support 

<http://www.insurekidsnow.gov>—the HHS/HCFA website for the CHIP programs 
<http://www.cbpp.org>the Center on Budget and Policy Priorities 


Thank you for your thoughtful consideration of this request. 
If you wish to speak with someone about this project, please call me at 609-951-5799. Either Elaine Arkin, a 
consultant to the Foundation, or I would be happy to speak with you. 


Sincerely 


Stuart Schear 
Senior Communications Officer 


National U.S. healthcare reform, many parts slated for implementation in 2013, will likely affect SCHIP (the subject of this RFP), but at this 
time it is unknown exactly in what ways or with what effects. 
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Background 


What if your firm manufactures cleansing products in multiple forms—deodorant bar, beauty bar, 
cream body wash, gel body wash—and your customers are not using the best form for their skin types 
and activity levels? You might use exploratory focus groups to determine what drivers motivate cus- 
tomers to select the form they choose. Given the dramatic growth in this market, you want to hear from 
women aged 16 to 50 and also from men aged 16 to 25. Also, you need to understand their trade-offs 
when choosing a specific form. 

You turn to a research specialist to conduct focus groups in three cities representative of the cat- 
egory market. Prior to meeting the six groups (two groups in each city; two consisting only of teens), 
researchers ask each participant to prepare two visual collages using pictures cut from magazines. One 
collage is to reflect the participant’s perceptions and experiences with each form (regardless of personal 
use experience). A second collage is to depict a month in the participant’s life. The Intro and Forms 
segments of the discussion guide below reference these creative exercises. 


Personal Cleansing Form Drivers Atlanta, Seattle, Phoenix 


INTRO (15 min) 
A. ALL ABOUT ME—name, family info, work, play, activities, interests. SHOW LIFE INTHE MONTH COLLAGE 
B. AT SOME POINT ASK: How often shower / bathe? 


Use fragrances / perfume? How often? 
Use scented or unscented deodorant, lotions, etc? 


FORMS (60 min) 
A. LISTED ON EASEL “DEODORANT BAR, BEAUTY BAR, CREAM BODY WASH, GEL BODY WASH” 


Here are the different forms of soaps available that we want to learn about. 
How many have ever used ? Still using or moved on / rejected? 
B. EASEL RESPONSES (BE SURE PICTURES ARE LABELED) Show and describe your picture collage (from 
homework), as you tell what you like / not, what associate w/ form. 
What else like? / why use? 
What not like about ? Why not using (more often)? 


How compare to other forms—advantages / disadvantages? 
What wish for this form . . . what would make it better / perfect for you? 


*This discussion guide was developed by Pam Hay, an independent qualitative consultant for more than 24 years with a career focused 
on consumable goods (personal care products, health and beauty aids, and OTC drugs). Her experiences include conducting focus 
groups, individual depth interviews, ethnography home visits, and multifunctional consumer direct processes for the purposes of con- 
cept development, advertising evaluation, insight exploration, consumer segmentation, and product development. 
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How / why begin to use? Specifically, what remember about form then? 
How find out about it? (ads, TV commercial, friends) What details remember about the ad—what show, 
who in it? 

REPEAT FOR ALL FORMS 


C. LINE UP THE FORMS—When you think about these different forms, are they basically the same—just a 
different form or do you think of these as different products with different results? Describe. 
D. EXPLAIN CHART—Line these attributes up in the order you think is best to worst for each of the attributes 
listed on the paper. 
CLEANLINESS / SKIN CARE / GERM KILL / DEODORANCY / LATHER / SCENT / VALUE 


Why put in this order? What experience / notice with this form for (attribute above)? 
What about the form makes that difference? 
How much do you care about (attributes above)? Why / not? Affect whether you'd buy it? 


E. SHOW EXAMPLES OF BRANDS WITH BOTH BAR AND BODY WASH— 
Oil of Olay / Dove / Lever 2000 / Dial 


So to summarize with some specific brands, whether you have tried or not, what would you expect to be the 
difference, if any, when using the bar of brand vs. the body wash? 


What difference, if any, in how they make you feel emotionally after bathing with the bar vs. body wash? 


BRANDS (30 min) 


A. Now let’s focus on different brands. Write your favorite on your name card. 
LINE UP EXAMPLES OF THOSE COMMONLY USED. 


B. How many of you have used ? How often / long ago? 


Why use / choose (at store)? What like (better) about ‘ 
Why NOT use (more often)? 


C. How many have tried Oil of Olay / Dove / Lever 2000 / Dial? Why / not (more recently)? 


What associate with (brand above)? What stand for? What makes it different / unique vs. other brands? 


SUMMARY (15 min) 


A. There are 3 basic considerations when choosing a soap—brand, form, price. Put them in order of what matters 
most, 2nd, 3rd. For example, you go to the store to buy, and your usual form isn’t available in your usual brand, etc. 
What would you buy? 


MAKE A CHART OF RESPONSES 


B. Now think about 3 benefits we’ ve discussed—skin care, scent, cleanliness. Put these in order of importance. 
Describe order. 


Why is more important than ? 


(MISC—TIME PERMITTING) 


C. What do you see / notice in the store when shopping this aisle? New things? Switch around? 


ADULTS—How many of you buy soap for other family members? Who? How do you decide which form / 
brand to choose for your husband / teen? 


D. TEENS ONLY—Let’s talk a bit more about how you learn about new types brands or versions of soaps. 
Where have you seen ads? 


(Mall / locker room / Channel 1/ dressing rooms / etc.) 
What do you remember about it? What show / say / what was the main idea? 


What do you think of celebrity endorsements? 


CONCLUSION 
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This appendix contains additional nonparametric tests of hypotheses to augment those described in 
Chapter 17. 


One-Sample Test 


Kolmogorov-Smirnov Test 


This test is appropriate when the data are at least ordinal and the research situation calls for a com- 
parison of an observed sample distribution with a theoretical distribution. Under these conditions, the 
Kolmogorov-Smimnov (KS) one-sample test is more powerful than the x? test and can be used for small 
samples when the x’ test cannot. The KS is a test of goodness of fit in which we specify the cumulative 
frequency distribution that would occur under the theoretical distribution and compare that with the 
observed cumulative frequency distribution. The theoretical distribution represents our expectations 
under H,. We determine the point of greatest divergence between the observed and theoretical distri- 
butions and identify this value as D (maximum deviation). From a table of critical values for D, we 
determine whether such a large divergence is likely on the basis of random sampling variations from 
the theoretical distribution. The value for D is calculated as follows: 


D = maximum |F, (X) — F, (X)| 
in which 
F(X) = the observed cumulative frequency distribution of a random sample of n observations. 


Where X is any possible score, F,(X) = k/n, where k = the number of observations equal 
to or less than X. 


F(X) = the theoretical frequency distribution under H,. 


We illustrate the KS test, with an analysis of the results of the dining club study, in terms of vari- 
ous class levels. Take an equal number of interviews from each class, but secure unequal numbers of 
people interested in joining. Assume class levels are ordinal measurements. The testing process is as 
follows (see accompanying table): 


Freshman Sophomore Junior Senior Graduate 
Number in each class 5 9 14 16 19 
F(X) 5/60 14/60 25/60 41/60 60/60 
F(X) 12/60 24/60 36/60 48/60 60/60 
|F (X) = F,(X)| 7/60 10/60 11/60 7/60 ) 
D = 11/60 = .183; n = 60 
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1. Null hypothesis. 
H,: There is no difference among student classes as to their intention of joining the dining club. 


H,: There is a difference among students in various classes as to their intention of joining the 
dining club. 
2. Statistical test. Choose the KS one-sample test because the data are ordinal measures and we 
are interested in comparing an observed distribution with a theoretical one. 


3. Significance level. a = .05,n = 60. 
4. Calculated value. D = maximum | F(X) — F, (X)|- 


5. Critical test value. We enter the table of critical values of D in the KS one-sample test (see 
Appendix d, Exhibit d-5) and learn that with a = .05, the critical value for D is 


6. Interpretation. The calculated value is greater than the critical value, indicating we should 
reject the null hypothesis. 


Two-Samples Tests 
Sign Test 


The sign test is used with matched pairs when the only information is the identification of the pair 
member that is larger or smaller or has more or less of some characteristic. Under H,, one would ex- 
pect the number of cases in which X, > X, to equal the number of pairs in which X, > X,. All ties are 
dropped from the analysis, and n is adjusted to allow for these eliminated pairs. This test is based on 
the binomial expansion and has a good power efficiency for small samples. 


Wilcoxon Matched-Pairs Test 


When you can determine both direction and magnitude of difference between carefully matched pairs, 
use the Wilcoxon matched-pairs test. This test has excellent efficiency and can be more powerful than 
the f-test in cases where the latter is not particularly appropriate. The mechanics of calculation are also 
quite simple. Find the difference score (d,) between each pair of values, and rank-order the differences 
from smallest to largest without regard to sign. The actual signs of each difference are then added to 
the rank values, and the test statistic T is calculated. T is the sum of the ranks with the less frequent 
sign. Typical of such research situations might be a study where husband and wife are matched, where 
twins are used, where a given subject is used in a before-after study, or where the outputs of two similar 
machines are compared. 

Two types of ties may occur with this test. When two observations are equal, the d score becomes 
zero, and we drop this pair of observations from the calculation. When two or more pairs have the same 
d value, we average their rank positions. For example, if two pairs have a rank score of 1, we assign the 
rank of 1.5 to each and rank the next largest difference as third. When 1 < 25, use the table of critical 
values (see Appendix d, Exhibit d-4). When n > 25, the sampling distribution of 7 is approximately 
normal with 


ee cae 
Standard deviation = o\ne us pen a 
The formula for the test is 
T- 4, 
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Suppose you conduct an experiment on the effect of brand name on quality perception. Ten subjects 
are recruited and asked to taste and compare two samples of a product, one identified as a well-known 
drink and the other as a new product being tested. In truth, however, the samples are identical. The 
subjects are then asked to rate the two samples on a set of scale items judged to be ordinal. Test these 
results for significance by the usual procedure. 


1. Null hypothesis. 
H,;: There is no difference between the perceived qualities of the two samples. 
H,: There is a difference in the perceived quality of the two samples. 


2. Statistical test. The Wilcoxon matched-pairs test is used because the study is of related samples 
in which the differences can be ranked in magnitude. 


3. Significance level. a = .05, with n = 10 pairs of comparisons minus any pairs with 
ad of zero. 


4. Calculated value. T equals the sum of the ranks with the less frequent sign. Assume we secure 
the following results: 


Rank with Less 


Branded Unbranded E Rank of d, Frequent Sign 

1 62 48 4 4 

2 37 32 5 5.5* 

3 50 52 =2 =2 2 

4 45 32 13 g) 

5 56 59 = 8) =6 3 

6 51 50 1 1 

7 40 28) 11 8 

8 59 54 5 5.5" 

@) 38 38 ¢) a 

10 40 32 8 if f=6 


*There are two types of tie situations. We drop out the pair with the type of tie shown by pair 9. Pairs 2 and 8 have a 
tie in rank of difference. In this case, we average the ranks and assign the average value to each. 


5. Critical test value. Enter the table of critical values of T with n = 9 (see Appendix d, 
Exhibit d-4) and find that the critical value with a = .05 is 6. Note that with this 
test, the calculated value must be smaller than the critical value to reject the null 
hypothesis. 


6. Interpretation. Since the calculated value is less than the critical value, reject the null 
hypothesis. 


Kolmogorov-Smirnov Two-Samples Test 


When a researcher has two independent samples of ordinal data, the Kolmogorov-Smirnov (KS) two- 
samples test is useful. Like the one-sample test, this two-samples test is concerned with the agreement 
between two cumulative distributions, but both represent sample values. If the two samples have been 
drawn from the same population, the cumulative distributions of the samples should be fairly close to 
each other, showing only random deviations from the population distribution. If the cumulative distri- 
butions show a large enough maximum deviation D, it is evidence for rejecting the H,. To secure the 
maximum deviation, one should use as many intervals as are available so as not to obscure the maxi- 
mum cumulative difference. 


>appendix e Nonparametric Significance Tests 


The two-samples KS formula is 
D = maximum |F,, (X) — F,, (X)| (two-tailed test) 
D = maximum |F,, (X) — F,, (X)| (one-tailed test) 
Dis calculated in the same manner as before, but the table for critical values for the numerator of D, K a 
(two-samples test) is presented in Appendix d, Exhibit d-6, when n, = n, and is less than 40 observa- 
tions. When n, and/or n, is larger than 40, D from Appendix d, Exhibit d-7, should be used. With this 
larger sample, it is not necessary that n, = n,. 

Here we use a different sample from a tobacco industry advertising study. Suppose the smoking 
classifications represent an ordinal scale (heavy smoker, moderate smoker, non-smoker), and you test 
these data with KS two-samples test for young and old age groups. Proceed as follows: 

1. Null hypothesis. 

H,;: There is no difference in ages of smokers and nonsmokers. 
H,: The older the person, the more likely he or she is to be a heavy smoker. 

2. Statistical test. The KS two-samples test is used because it is assumed the data are ordinal. 

3. Significance level. a = .05.n, =n, = 34. 

4. Calculated value. See the one-sample calculation (KS test) and compare with the table below. 


5. Critical test value. We enter Appendix d, Exhibit d-6, with n = 34 to find that K, = 11 when 
p = .05 for a one-tailed distribution. 


Heavy Smoker Moderate Smoker Nonsmoker 
IF , (X 12/34 21/34 34/34 
F(X) 4/34 10/34 34/34 
d=K, 8/34 11/34 O 


i Din 


6. Interpretation. Since the critical value equals the largest calculated value, we reject the null 
hypothesis. 


Mann-Whitney U Test 


This test is also used with two independent samples if the data are at least ordinal; it is an alternative 
to the t-test without the latter’s limiting assumptions. When the larger of the two samples is 20 or less, 
there are special tables for interpreting U; when the larger sample exceeds 20, a normal curve approxi- 
mation is used. 

In calculating the U test, treat all observations in a combined fashion and rank them, algebraically, 
from smallest to largest. The largest negative score receives the lowest rank. In case of ties, assign the 
average rank as in other tests. With this test, you can also test samples that are unequal. After the rank- 
ing, the rank values for each sample are totaled. Compute the U statistic as follows: 


n(n, + 1) 
U=nn,+— 5 ; 
or 
n(n, — 1) 
U=nn, + 2 5) R, 
in which 


n, = number in sample | 
n, = number in sample 2 


R, = sum of ranks in sample 1 
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With this equation, you can secure two U values, one using R, and the second using R,. For testing 
purposes, use the smaller U. 

An example may help to clarify the U statistic calculation procedure. Let’s consider the sales train- 
ing example with the ¢ distribution discussion. Recall that salespeople with training method A averaged 
higher sales than salespeople with training method B. Although these data are ratio measures, one still 
might not want to accept the other assumptions that underlie the f-test. What kind of a result could be 
secured with the U test? While the U test is designed for ordinal data, it can be used with interval and 
ratio measurements. 


1. Null hypothesis. 
H,;: There is no difference in sales results produced by the two training methods. 
H,: Training method A produces sales results superior to the results of method B. 


2. Statistical test. The Mann-Whitney U test is chosen because the measurement is at least ordinal, 
and the assumptions under the parametric f-test are rejected. 


3. Significance level. a = .05 (one-tailed test). 


4. Calculated value. 


Sales per Week per Salesperson 


Training Method A Rank Training Method B 
1,500 15 1,340 10 
1,540 16 1,300 8.5 
1,860 22 1,620 18 
1,230 6 1,070 8 
1,370 12 1,210 5 
1,550 17 1,170 4 
1,840 21 1,770 20 
1,250 i 950 1 
1,300 8.5 1,380 13 
1,350 chi 1,460 14_ 
1,710 19 1,030 2 

R, = 154.5 R, = 98.5 
U = (14)(11) 4 AE 154.5 = 32.5 U = (11)(41) 4 a 98.5 = 88.5 


5. Critical test value. Enter Appendix d, Exhibit d-9, with n, = n, = 11, and find a critical value 
of 34 for a = 0.5, one-tailed test. Note that with this test, the calculated value must be smaller 
than the critical value to reject the null hypothesis. 


6. Interpretation. Since the calculated value is smaller than the critical value (34 > 32.5), reject 
the null hypothesis and conclude that training method A is probably superior. 


Thus, one would reject the null hypothesis at « = .05 in a one-tailed test using either the t- or the U 
test. In this example, the U test has approximately the same power as the parametric test. 

When n > 20 in one of the samples, the sampling distribution of U approaches the normal distribu- 
tion with 


nn, 


2 


Mean = p,, = 


n, +n, +1) 
12 


Standard deviation o,, = 4 ee 
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and 


Other Nonparametric Tests 


Other tests are appropriate under certain conditions when testing two independent samples. When the 
measurement is only nominal, the Fisher exact probability test may be used. When the data are at least 
ordinal, use the median and Wald-Wolfowitz runs tests. 


k-Samples Tests 


You can use tests more powerful than x’ with data that are at least ordinal in nature. One such test is an 
extension of the median test mentioned earlier. We illustrate here the application of a second ordinal 
measurement test known as the Kruskal-Wallis one-way analysis of variance. 


Kruskal-Wallis Test 


This is a generalized version of the Mann-Whitney test. With it we rank all scores in the entire pool of 
observations from smallest to largest. The rank sum of each sample is then calculated, with ties being 
distributed as in other examples. We then compute the value of H as follows: 


k 
_ 2 T; 
H NED ered) 


where 
T= sum of ranks in column j 
n= number of cases in jth sample 
N= =w, = total number of cases 


k = number of samples 


When there are a number of ties, it is recommended that a correction factor (C) be calculated and used 
to correct the H value as follows: 


Lai -1) 


= 1 
c N?-N 
where 
G = number of sets of tied observations 
t, = number tied in any set i 
H'= H/C 


To secure the critical value for H’, use the table for the distribution of x? (see Appendix d, Exhibit d-3), 
and enter it with the value of H’ and d.f. = k — 1. 

To illustrate the application of this test, use the price discount experiment problem. The data and 
calculations are shown in Exhibit c-1 and indicate that, by the Kruskal-Wallis test, one again barely 
fails to reject the null hypothesis with a = .05. 
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>Exhibit c-1 Kruskal-Wallis One-Way Analysis of Variance (price differentials) 


One Cent Three Cents Five Cents 

Rank Rank Rank 
6 1 8 5 g) 8.5 
if 2.5 g) 8.5 g) 8.5 

8 5 8 5 dd 14 
i 245) 10 les) 10 lies) 

9 8.5 11 14 14 18 
11 14 13 16.5 13 16.5 
T, = 33.5 60.5 77.0 


T = 33.5 + 60.5 + 77.0 


=171 
121335? + 60.52 + 772 
ce EET 6 3(18 + 1) 
_ 412 [dees + 3,660.25 + 5,929) ee 
340 6 
_ 0.0351 10.7115) = 
6 
Hs 66 
ae E (2)° — 2] + 218) — 3] + 4[(42 4] 
18° — 18 
, — 18 +48 + 60 
5814 
= 978 


Selected Statistical Tables 


d-1 
d-2 
d-3 
d-4 
d-5 
d-6 
d-7 


d-8 
d-9 
d-10 


Areas of the Standard Normal Distribution 

Critical Values of t for Given Probability Levels 

Critical Values of the Chi-Square Distribution 

Critical Values of T in the Wilcoxon Matched-Pairs Test 

Critical Values of D in the Kolmogorov-Smirnov One-Sample Test 

Critical Values of K,, in the Kolmogorov-Smirnov Two-Samples Test (small samples) 
Critical Values of D in the Kolmogorov-Smirnov Two-Samples Test for Large Samples 
(two-tailed) 

Critical Values of the F Distribution for a = .05 

Partial Table of Critical Values of U in the Mann-Whitney Test 

Random Numbers 
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Exhibit d-1 Areas of the Standard Normal Distribution 0 Z 


Second Decimal Place in z 


0.03 0.04 0.05 


0.0 0.0000 0.0040 0.0080 0.0120 0.0160 0.0199 0.0239 0.0279 0.0319 0.0359 
0.1 0.0398 0.0438 0.0478 0.0517 0.0557 0.0596 0.0636 0.0675 0.0714 0.0753 
0.2 0.0793 0.0832 0.0871 0.0910 0.0948 0.0987 0.1026 0.1064 0.1108 0.1144 
0.3 Ole) 0.1217 0.1255 0.1293 0.1331 0.1368 0.1406 0.1443 0.1480 0.1517 
0.4 0.1554 0.1591 0.1628 0.1664 0.1700 0.1736 0.1772 0.1808 0.1844 0.1879 
0.5 0.1915 0.1950 0.1985 0.2019 0.2054 0.2088 0.2128 0.2157 0.2190 0.2224 


0.6 0.2257 0.2291 0.2324 0.2357 0.2389 0.2422 0.2454 0.2486 0.2517 0.2549 
0.7 0.2580 0.2614 0.2642 0.2673 0.2704 0.2734 0.2764 0.2794 0.2828 0.2852 
0.8 0.2881 0.2910 0.2939 0.2967 0.2995 0.3023 0.3051 0.3078 0.3106 0.3133 
0.9 0.3159 0.3186 0.3212 0.3238 0.3264 0.3289 0.3315 0.3340 0.3365 0.3389 
1.0 0.3413 0.3438 0.3461 0.3485 0.3508 0.3531 0.3554 0.3577 0.3599 0.3621 


1 0.3643 0.3665 0.3686 0.3708 0.3729 0.3749 0.3770 0.3790 0.3810 0.3830 

2 0.3849 0.3869 0.3888 0.3907 0.3925 0.3944 0.3962 0.3980 0.3997 0.4015 
as) 0.4032 0.4049 0.4066 0.4082 0.4099 0.4115 0.4131 0.4147 0.4162 0.4177 

4 

5 


0.4192 0.4207 0.4222 0.4236 0.4251 0.4265 0.4279 0.4292 0.4306 0.4319 
0.4332 0.4345 0.4357 0.4370 0.4382 0.4394 0.4406 0.4418 0.4429 0.4444 


6 0.4452 0.4463 0.4474 0.4484 0.4495 0.4505 0.4515 0.4525 0.4535 0.4545 
all 0.4554 0.4564 0.4573 0.4582 0.4591 0.4599 0.4608 0.4616 0.4625 0.4633 
8 0.4641 0.4649 0.4656 0.4664 0.4671 0.4678 0.4686 0.4693 0.4699 0.4706 
i) 0.4713 0.4719 0.4726 0.4732 0.4738 0.4744 0.4750 0.4756 0.4761 0.4767 
2.0 0.4772 0.4778 0.4783 0.4788 0.4798 0.4798 0.4803 0.4808 0.4812 0.4817 


ral 0.4821 0.4826 0.4830 0.4834 0.4838 0.4842 0.4846 0.4850 0.4854 0.4857 
Deed 0.4861 0.4864 0.4868 0.4871 0.4875 0.4878 0.4881 0.4884 0.4887 0.4890 
2.3 0.4893 0.4896 0.4898 0.4901 0.4904 0.4906 0.4909 0.4914 0.4913 0.4916 
2.4 0.4918 0.4920 0.4922 0.4925 0.4927 0.4929 0.4931 0.4932 0.4934 0.4936 
2S) 0.4938 0.4940 0.4941 0.4943 0.4945 0.4946 0.4948 0.4949 0.4951 0.4952 


2.6 0.4953 0.4955 0.4956 0.4957 0.4959 0.4960 0.4961 0.4962 0.4963 0.4964 
2olf 0.4965 0.4966 0.4967 0.4968 0.4969 0.4970 0.4971 0.4972 0.4973 0.4974 
2.8 0.4974 0.4975 0.4976 0.4977 0.4977 0.4978 0.4979 0.4979 0.4980 0.4981 
Zn9) 0.4981 0.4982 0.4982 0.4983 0.4984 0.4984 0.4985 0.4985 0.4986 0.4986 
3.0 0.4987 0.4987 0.4987 0.4988 0.4988 0.4989 0.4989 0.4989 0.4990 0.4990 


3.1 0.4990 0.4991 0.4991 0.4991 0.4992 0.4992 0.4992 0.4992 0.4993 0.4993 
3.2 0.4998 0.4993 0.4994 0.4994 0.4994 0.4994 0.4994 0.4995 0.4995 0.4995 
3.3 0.4995 0.4995 0.4995 0.4996 0.4996 0.4996 0.4996 0.4996 0.4996 0.4997 
3.4 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4997 0.4998 


3.5 0.4998 

4.0 0.49997 

4.5 0.499997 

5.0 0.4999997 
6.0 0.499999999 
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>Exhibit d-2 Critical Values of t for Given Probability Levels 


Level of Significance for One-Tailed Test 


.05 .025 01 .005 


Level of Significance for Two-Tailed Test 


-10 .05 .02 


636.619 
31.598 
12.941 

8.610 
6.859 


Source: Abridged from Table Ill of Fisher and Yates, Statistical Tables for Biological, Agricultural, 
and Medical Research, 6th ed., published by Oliver and Boyd Ltd., Edinburgh, 1963. By permission of 
the publishers. 
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>Exhibit d-3 Critical Values of the Chi-Square Distribution 


0 Critical 
value 


Probability 


.02 


Source: Abridged from Table IV of Fisher and Yates, Statistical Tables for Biological, 
Agricultural, and Medical Research, 6th ed., published by Oliver and Boyd Ltd., 
Edinburgh, 1963. By permission of the publishers. 
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>Exhibit d-4 Critical Values of T in the Wilcoxon Matched-Pairs Test 


Level of Significance for One-Tailed Test 


01 .005 


Level of Significance for Two-Tailed Test 


Source: Adapted from Table 1 of F. Wilcoxon, Some Rapid Approximate Statistical Procedures 
(New York: American Cyanamid Company, 1949), p. 13, with the kind permission of the publisher. 


>Exhibit d-5 Critical Values of D in the Kolmogorov-Smirnov One-Sample Test 


Level of Significance for D = Maximum |F,(X) — S,(X)| 


Sample Size n .20 15 .10 .05 .01 


25) : 
30 29 
35 ih 
1.63 
Over 35 SS 
VN 


Source: F. J. Massey uJr., “The Kolmogorov-Smirnov Test for Goodness of Fit,” Journal of the American 
Statistical Association 46, p. 70. Adapted with the kind permission of the publisher. 
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>Exhibit d-6 Critical Values of K,, in the Kolmogorov-Smirnov 
Two-Samples Test (small samples) 


One-Tailed Test Two-Tailed Test 


Source: One-tailed test— abridged from |. A. Goodman, “Kolmogorov-Smirnov Tests for 
Psychological Research,” Psychological Bulletin 51 (1951), p. 167, copyright (1951) by the 
American Psychological Association. Reprinted by permission. Two-tailed test—derived from 
Table 1 of F. J. Massey Jr., “The Distribution of the Maximum Deviation between Two Sample 
Cumulative Step Functions,” Annals of Mathematical Statistics 23 (1951), pp. 126-27, with 
the kind permission of the publisher. 
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>Exhibit d-7 Critical Values of D in the Kolmogorov-Smirnov Two-Samples Test 
for Large Samples (two-tailed) 


Value of D So Large as to Call for Rejection of H, 
at the Indicated Level of Significance, Where 
Level of Significance D = Maximum |S, ,(X) = S,(X)| 


Source: Adapted from N. Smirnov, “Table for Estimating the Goodness of Fit of Empirical Distribution,” 
Annals of Mathematical Statistics 18 (1948), pp. 280-81, with the kind permission of the publisher. 
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>Exhibit d-8 Critical Values of the F Distribution fora = .05 0 


Degrees of Freedom for Numerator 


Denominator 


4.08 3.23 2.84 2.61 2.45 2.34 2.25 2.18 
60 4.00 3.15 2.76 2.53 2.37 2.25 oil 2.10 2.04 
120 6:92 3.07 2.68 2.45 DS) Zane 2.09 2.02 1.96 
% 3.84 3.00 2.60 2.37 Be) 2.10 2.01 1.94 1.88 


Source: Reprinted by permission from Statistical Methods by George W. Snedecor and William G. Cochran, 
6th edition, © 1967 by lowa State University Press, Ames, lowa. 
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Degrees of Freedom for Numerator (cont’d) 


20 24 30 40 


_ 
£ 
© 
= 
E 
6 
= 
o 
() 


60 1-88) deo? 1.84 1.75 1.70 1.65 1.59 1.53 1.47 1.39 
120 1) 1.83 1.05) 1.66 1.61 1.55 1.50 1.43 1.35 1.25 
0 1.83 1.75 hOv EO 1.62 1.46 <3) 1.32 1.22 1.00 
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Exhibit d-9 Partial Table of Critical Values of U in the Mann-Whitney Test 


Critical Values for One-Tailed Test at a = .025 or Two-Tailed Test at a = .05 


12 15 16 17 


Source: Abridged from D. Auble, “Extended Tables from the Mann-Whitney Statistic,” Bulletin of the Institute of Educational Research at 
Indiana University 1, no. 2, reprinted with permission. For tables for other-size samples, consult this source. 


>Exhibit d-10 Random Numbers 


97446 
15453 
69995 
69726 
23604 
13640 
SIO TAS) 
71068 
55019 
20879 
00380 
68142 
42667 
07756 
45221 
20140 
54922 
48340 
23603 
12548 
15251 
65548 
48488 
02862 
67929 
73237 
13788 
43877 
14047 
88383 
29567 
20508 
64178 
26373 
10083 
11683 
56548 
35292 
51329 
51860 
23886 
80028 
83946 
00000 
90615 
42003 
86135 
54436 
14545 
43925 


Source: The Rand Corporation, A Million Random Digits with 100,000 Normal Deviates (Glencoe, IL: Free Press, 1955), p. 225. 


30328 
75591 
77086 
58696 
31948 
17233 
09199 
19459 
79001 
50235 
11595 
40800 
89566 
78430 
31130 
77481 
25436 
30832 
84387 
01033 
87584 
59670 
76170 
15665 
06754 
07607 
20327 
66485 
08369 
46755 
16379 
60995 
76768 
51147 
47656 
14347 
76293 
47291 
87645 
19180 
01257 
82310 
46858 
41586 
12848 
10738 
26174 
10828 
72034 
49879 


05262 
60540 
55217 
27272 
16926 
58650 
51169 
32339 
34442 
17389 
49372 
20527 
20440 
45576 
44312 
61686 
33804 
72209 
20416 
22974 
12942 
57355 
46282 
62159 
45842 
31615 
07960 
40825 
56414 
51342 
41994 
41539 
T5747 
90362 
59241 
04369 
50904 
82610 
51623 
39324 
07945 
43989 
09164 
25972 
23376 
55835 
07834 
41212 
32131 
13339 


17371 
CUS 
53721 
38148 
26360 
47819 
94892 
10124 
16335 
25260 
95214 
WREW\2 
57230 
86596 
63534 
82836 
51907 
07644 
88084 
59596 
03771 
18874 
16427 
15159 
66365 
04892 
95917 
45923 
78533 
13505 
65947 
26396 
32854 
93309 
73630 
98719 
88579 
AG 
08971 
68483 
CVITAS 
09242 
18858 
25356 
29458 
48218 
17007 
19836 
38783 
78773 


13523 
09485 
85713 
52521 
76957 
24935 
34271 
13012 
06428 
34039 
98529 
14166 
35356 
56720 
47741 
41058 
73223 
52747 
33103 
92087 
91413 
63601 
41693 
69576 
80848 
50989 
79112 
74410 
76378 
55324 
58926 
99825 
32893 
13175 
99200 
75005 
24621 
43965 
50704 
78650 
31243 
15056 
12672 
54260 
48239 
23204 
97938 
89476 
58588 
95626 


62057 
27632 
27854 
73807 
99925 
28670 
22068 
79706 
52873 
99967 
46593 
84948 
01884 
65529 
02465 
41331 
66423 
40751 
41511 
02116 
75652 
$5111 
04506 
20328 
15262 
87347 
01398 
69693 
44204 
52950 
50953 
25652 
61152 
66385 
94672 
43633 
94291 
31802 
82395 
74750 
87167 
48250 
55190 
95691 
37628 
19188 
96728 
53685 
47499 
67119 
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44349 
05477 
41981 
29685 
86045 
33415 
13923 
07611 
65316 
48044 
77046 
11748 
79921 
44211 
50629 
04290 
68706 
06808 
59391 
63524 
19468 
07278 
80979 
68873 
55144 
14393 
26381 
76959 
71498 
22244 
09388 
28089 
58565 
57822 
59785 
24125 
56881 
98444 
33916 
64893 
42829 
04529 
02820 
99431 
59265 
13556 
15689 
28085 
50945 
93023 


85884 
99154 
88981 
49152 
11617 
77202 
53535 
52600 
01480 
05067 
27176 
69540 
94772 
18447 
94983 
61212 
36589 
85349 
71600 
00627 
83889 
32560 
26654 
28152 
37816 
21165 
41377 
70973 
68861 
28028 
00405 
57224 
33128 
31138 
95449 
30532 
35062 
88929 
95859 
58042 
44601 
96941 
45861 
89903 
50152 
06610 
11544 
22878 
97045 
96832 


94555 
78720 
90041 
20309 
32777 
92492 
56358 
83088 
72204 
69284 
39668 
84288 
29882 
53921 
05984 
60294 
45267 
18005 
35091 
41778 
98531 
40028 
62159 
66087 
08421 
68169 
33549 
26343 
31042 
73486 
29874 
35222 
33354 
12893 
SIATS) 
54830 
48765 
54383 
99788 
82878 
08769 
48190 
29104 
22306 
30340 
77667 
89186 
71868 
42357 
09757 
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23288 
10323 
20878 
58734 
38670 
40290 
50258 
26829 
39494 
53867 
20566 
37211 
24695 
92722 
88375 
95954 
35327 
52323 
$2722 
24392 
91529 
36079 
83017 
39405 
30071 
70788 
19754 
63781 
81873 
98797 
44954 
58922 
16056 
68607 
25488 
95387 
22078 
93144 
97885 
20619 
26417 
69644 
75386 
43863 
40713 
88068 
41252 
35048 
53536 
98545 
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385.html). 


PullQuote 

“Don’t Be in the 4%: New research Reveals that 96% of Companies 
Are Leveraging VoC to Improve Business Performance,” Peppers & 
Rogers Group, 2012, accessed April 21, 2012 (http://www 
.ltolmedia.com/downloads/vovici_prg_voc%20white%20paper_ 
FINAL_021712.pdf?%E2%80%9D). 
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Lexus/Team One Advertising 

Mark Miller, associate director, strategic planning, Team One Advertis- 
ing, interviewed July 9 and October 5, 2002. 


Public Speaking Jitters 

Patricia Fripp, CSP, CPAE, award-winning keynote speaker and speech 
coach, author of Get What You Want!, and past president of the 
National Speakers Association (http://www.fripp.com/). 


PullQuote 
Mark Hill, “Oral Presentation Advice.” accessed April 20, 2012 (http:// 
pages.cs.wisc.edu/~markhill/conference-talk.html). 


a priori contrasts a special class of tests used in conjunction 
with the F test that is specifically designed to test the hypoth- 
eses of the experiment or study (in comparison to post hoc or 
unplanned tests). 

accuracy the degree to which bias is absent from the 
sample—the underestimators and the overestimators are 
balanced among members of the sample (i.e., no systematic 
variance). 

action research a methodology with brainstorming followed by 
sequential trial-and-error to discover the most effective solu- 
tion to a problem; succeeding solutions are tried until the de- 
sired results are achieved; used with complex problems about 
which little is known. 

active factors those independent variables (IV) the researcher 
can manipulate by causing the subject to receive one treat- 
ment level or another. 

activity analysis see process analysis. 

administrative question a measurement question that identifies 
the participant, interviewer, interview location, and condi- 
tions (nominal data). 

alternative hypothesis (H,) that a difference exists between 
the sample parameter and the population statistic to which it 
is compared; the logical opposite of the null hypothesis used 
in significance testing. 

ambiguities and paradoxes a projective technique (imagina- 
tion exercise) in which participants imagine a brand applied 
to a different product (e.g., a Tide dog food or Marlboro ce- 
real), and then describe its attributes and position. 

analogy a rhetorical device that compares two different things 
to highlight a point of similarity. 

analysis of variance (ANOVA) tests the null hypothesis that 
the means of several independent populations are equal; test 
statistic is the F ratio; used when you need k-independent- 
samples tests. 

applied research research that addresses existing problems or 
opportunities. 

arbitrary scales universal practice of ad hoc scale development 
used by instrument designers to create scales that are highly 
specific to the practice or object being studied. 

area chart a graphical presentation that displays total fre- 
quency, group frequency, and time series data; a.k.a. stratum 
chart or surface chart. 

area sampling a cluster sampling technique applied to a popu- 
lation with well-defined political or natural boundaries; pop- 
ulation is divided into homogeneous clusters from which a 
single-stage or multistage sample is drawn. 

argument statement that explains, interprets, defends, chal- 
lenges, or explores meaning. 

artifact correlations occur when distinct subgroups in the data 
combine to give the impression of one. 

association the process used to recognize and understand pat- 
terns in data and then used to understand and exploit natural 
patterns. 

asymmetrical relationship a relationship in which we postu- 
late that change in one variable (IV) is responsible for change 
in another variable (DV). 
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attitude a learned, stable predisposition to respond to oneself, 
other persons, objects, or issues in a consistently favorable or 
unfavorable way. 

audience characteristics and background of the people or groups 
for whom the secondary source was created; one of the five 
factors used to evaluate the value of a secondary source. 

audience analysis an analysis of the attendees at a presentation 
through advance conversations or psychological profiling of 
age, size, education/knowledge level, experience, gender, 
diversity, company culture, decision-making roles, and indi- 
vidual attitudes, needs, and motivations. 

auditory learners audience members who learn through listen- 
ing; represent about 20 to 30 percent of the audience; im- 
plies the need to include stories and examples in research 
presentations. 

authority the level of data and the credibility of a source as in- 
dicated by the credentials of the author and publisher; one of 
five factors used to evaluate the value of a secondary source. 

authority figure a projective technique (imagination exercise) 
in which participants are asked to imagine that the brand or 
product is an authority figure and to describe the attributes 
of the figure. 

automatic interaction detection (AID) a data partitioning pro- 
cedure that searches up to 300 variables for the single best 
predictor of a dependent variable. 

average linkage method evaluates the distance between two 
clusters by first finding the geometric center of each cluster 
and then computing distances between the two centers. 


backward elimination sequentially removing the variable from 
a regression model that changes R? the least; see also forward 
selection and stepwise selection. 

balanced rating scale has an equal number of categories above 
and below the midpoint or an equal number of favorable/ 
unfavorable response choices. 

band see prediction and confidence bands. 

bar chart a graphical presentation technique that represents fre- 
quency data as horizontal or vertical bars; vertical bars are 
most often used for time series and quantitative classifica- 
tions (histograms, stacked bar, and multiple-variable charts 
are specialized bar charts). 

bar code technology employing labels containing electronically 
read vertical bar data codes. 

basic research see pure research. 

Bayesian statistics uses subjective probability estimates based 
on general experience rather than on data collected. (See 
“Decision Theory Problem” at the Online Learning Center.) 

benefit chain see laddering. 

beta weights standardized regression coefficients in which the 
size of the number reflects the level of influence X exerts on Y. 

bibliography (bibliographic database) a secondary source 
that helps locate a book, article, photograph, etc. 

bivariate correlation analysis a statistical technique to assess 
the relationship of two continuous variables measured on an 
interval or ratio scale. 


bivariate normal distribution data are from a random sample 
in which two variables are normally distributed in a joint 
manner. 

blind when participants do not know if they are being exposed 
to the experimental treatment. 

boxplot an EDA technique; a visual image of the variable’s 
distribution location, spread, shape, tail length, and outliers; 
a.k.a. box-and-whisker plot. 

branched question a measurement question sequence deter- 
mined by the participant’s previous answer(s); the answer 
to one question assumes other questions have been asked or 
answered and directs the participant to answer specific ques- 
tions that follow and skip other questions; branched questions 
determine question sequencing. 

brand mapping a projective technique (type of semantic map- 
ping) where participants are presented with different brands 
and asked to talk about their perceptions, usually in relation 
to several criteria. They may also be asked to spatially place 
each brand on one or more semantic maps. 

buffer question a neutral measurement question designed 
chiefly to establish rapport with the participant (usually nom- 
inal data). 

business intelligence system (BIS) a system of ongoing in- 
formation collection about events and trends in the techno- 
logical, economic, political and legal, demographic, cultural, 
social, and competitive arenas. 

business research a systematic inquiry that provides informa- 
tion to guide business decisions; the process of determining, 
acquiring, analyzing and synthesizing, and disseminating rele- 
vant business data, information, and insights to decision mak- 
ers in ways that mobilize the organization to take appropriate 
actions that, in turn, maximize organizational performance. 


callback procedure involving repeated attempts to make con- 
tact with a targeted participant to ensure that the targeted par- 
ticipant is reached and motivated to participate in the study. 

cartoons or empty balloons a projective technique where 
participants are asked to write the dialog for a cartoonlike 
picture. 

case the entity or thing the hypothesis talks about. 

case study (case history) a methodology that combines indi- 
vidual and (sometimes) group interviews with record anal- 
ysis and observation; used to understand events and their 
ramifications and processes; emphasizes the full contextual 
analysis of a few events or conditions and their interrelations 
for a single participant; a type of preexperimental design 
(one-shot case study). 

categorization for this scale type, participants put themselves 
or property indicants in groups or categories; also, a process 
for grouping data for any variable into a limited number of 
categories. 

causal-explanatory study a study that is designed to determine 
whether one or more variables explain the causes or effects of 
one or more outcome (dependent) variables. 

causal hypothesis see explanatory hypothesis. 

causal-predictive study a study that is designed to predict with 
regularity how one or more variables cause or affect one or 
more outcome (dependent) variables to occur. 
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causal study research that attempts to reveal a causal relation- 
ship between variables. (A produces B or causes B to occur.) 

causation situation where one variable leads to a specified ef- 
fect on the other variable. 

cell in a cross-tabulation, a subgroup of the data created by the 
value intersection of two (or more) variables; each cell con- 
tains the count of cases as well as the percentage of the joint 
classification. 

census acount of all the elements in a population. 

central limit theorem the sample means of repeatedly drawn 
samples will be distributed around the population mean; for 
sufficiently large samples (i.e., n = 30+), approximates a 
normal distribution. 

central tendency a measure of location, most commonly the 
mean, median, and mode. 

central tendency (error of) an error that results because the 
participant is reluctant to give extreme judgments, usually 
due to lack of knowledge. 

centroid a term used for the multivariate mean scores in 
MANOVA. 

checklist a measurement question that poses numerous alter- 
natives and encourages multiple unordered responses; see 
multiple-choice, multiple-response scale. 

chi-square-based measures tests to detect the strength of the 
relationship between the variables tested with a chi-square 
test: phi, Cramer’s V, and contingency coefficient C. 

chi-square (x?) test a test of significance used for nominal and 
ordinal measurements. 

children’s panel a series of focus group sessions in which the 
same child may participate in up to three groups in one year, 
with each experience several months apart. 

chronologic interviewing see sequential interviewing. 

clarity a design principle that advocates the use of visual tech- 
niques that allow the audience to perceive meaning from the 
location of elements. 

classical statistics an objective view of probability in which 
the hypothesis is rejected, or not, based on the sample data 
collected. 

classification question a measurement question that provides 
sociological-demographic variables for use in grouping par- 
ticipants’ answers (nominal, ordinal, interval, or ratio data). 

closed question/response a measurement question that presents 
the participant with a fixed set of choices (nominal, ordinal, 
or interval data). 

cluster analysis identifies homogeneous subgroups of study 
objects or participants and then studies the data by these 
subgroups. 

cluster sampling a sampling plan that involves dividing the 
population into subgroups and then draws a sample from 
each subgroup, a single-stage or multistage design. 

clustering a technique that assigns each data record to a group 
or segment automatically by clustering algorithms that iden- 
tify the similar characteristics in the data set and then parti- 
tion them into groups. 

clutter verbal behaviors in a presentation that distract the audi- 
ence; includes repetition of fillers such as “ah,” “um,” “you 
know,” or “like.” 

code of ethics an organization’s codified set of norms or stan- 
dards of behavior that guide moral choices about research 
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behavior; effective codes are regulative, protect the public 
interest, are behavior-specific, and are enforceable. 

codebook the coding rules for assigning numbers or other sym- 
bols to each variable; a.k.a. coding scheme. 

coding assigning numbers or other symbols to responses so 
that they can be tallied and grouped into a limited number 
of categories. 

coefficient of determination (r?) the amount of common vari- 
ance in X and Y, two variables in regression; the ratio of the 
line of best fit’s error over that incurred by using the mean 
value of Y. 

collinearity occurs when two independent variables are highly 
correlated; causes estimated regression coefficients to fluctu- 
ate widely, making interpretation difficult. 

communality in factor analysis, the estimate of the variance in 
each variable that is explained by the factors being studied. 

communication approach a study approach involving ques- 
tioning or surveying people (by personal interview, tele- 
phone, mail, computer, or some combination of these) and 
recording their responses for analysis. 

communication study the researcher questions the participants 
and collects their responses by personal or impersonal means. 

comparative scale a scale in which the participant evaluates 
an object against a standard using a numerical, graphical, or 
verbal scale. 

compatibility a design principle that encourages matching vi- 
sual techniques that comprise the form of the message to its 
content and the meaning of that content. 

component sorts a projective technique in which participants 
are presented with flash cards containing component features 
and asked to create new combinations. 

computer-administered telephone survey a telephone survey 
via voice-synthesized computer questions; data are tallied 
continuously. 

computer-assisted personal interview (CAPI) a personal, 
face-to-face interview (IDI) with computer-sequenced ques- 
tions, employing visualization techniques; real-time data 
entry possible. 

computer-assisted self-interview (CASI) computer-delivered 
survey that is self-administered by the participant. 

computer-assisted telephone interview (CATI) a telephone in- 
terview with computer-sequenced questions and real-time data 
entry; usually in a central location with interviewers in acousti- 
cally isolated interviewing carrels; data are tallied continuously. 

concealment a technique in an observation study in which the 
observer is shielded from the participant to avoid error caused 
by observer’s presence; this is accomplished by one-way mir- 
rors, hidden cameras, hidden microphones, etc. 

concept a bundle of meanings or characteristics associated with 
certain concrete, unambiguous events, objects, conditions, or 
situations. 

conceptual scheme the interrelationships between concepts and 
constructs. 

concordant when a participant that ranks higher on one ordinal 
variable also ranks higher on another variable, the pairs of 
variables are concordant. 

confidence interval the combination of interval range and de- 
gree of confidence. 

confidence level the probability that the results will be correct. 


confidentiality a privacy guarantee to retain validity of the re- 
search, as well as to protect participants. 

confirmatory data analysis an analytical process guided by 
classical statistical inference in its use of significance and 
confidence. 

confounding variables (CFV) two or more variables that are 
confounded when their effects on a response variable cannot 
be distinguished from each other. 

conjoint analysis measures complex decision making that re- 
quires multiattribute judgments; uses input from nonmetric 
independent variables to secure part-worths that represent the 
importance of each aspect of the participant’s overall assess- 
ment; produces a scale value for each attribute or property. 

consensus scaling scale development by a panel of experts 
evaluating instrument items based on topical relevance and 
lack of ambiguity. 

constant-sum scale the participant allocates points to more 
than one attribute or property indicant, such that they total to 
100 or 10; a.k.a. fixed-sum scale. 

construct a definition specifically invented to represent an ab- 
stract phenomenon for a given research project. 

construct validity see validity, construct. 

content analysis a flexible, widely applicable tool for measuring 
the semantic content of a communication—including counts, 
categorizations, associations, interpretations, etc. (e.g., used 
to study the content of speeches, ads, newspaper and maga- 
zine editorials, focus group and IDI transcripts); contains four 
types of items: syntactical, referential, propositional, and the- 
matic; initial process is done by computer. 

content validity see validity, content. 

contingency coefficient C a measure of association for nomi- 
nal, nonparametric variables; used with any-size chi-square 
table, the upper limit varies with table sizes; does not provide 
direction of the association or reflect causation. 

contingency table a cross-tabulation table constructed for sta- 
tistical testing, with the test determining whether the classifi- 
cation variables are independent. 

contrast a design principle that advocates using high-contrast 
techniques to quickly draw audience attention to the main 
point. 

control the ability to replicate a scenario and dictate a particu- 
lar outcome; the ability to exclude, isolate, or manipulate the 
influence of a variable in a study; a critical factor in inference 
from an experiment, implies that all factors, with the excep- 
tion of the independent variable (IV), must be held constant 
and not confounded with another variable that is not part of 
the study. 

control dimension in quota sampling, a descriptor used to 
define the sample’s characteristics (e.g., age, education, 
religion). 

control group a group of participants that is not exposed to the 
independent variable being studied but still generates a mea- 
sure for the dependent variable. 

control variable a variable introduced to help interpret the rela- 
tionship between variables. 

controlled test market real-time test of a product through arbi- 
trarily selected distribution partners. 

controlled vocabulary carefully defined subject hierarchies 
used to search some bibliographic databases. 


convenience sample nonprobability sample in which element 
selection is based on ease of accessibility. 

convenience sampling nonprobability sampling in which re- 
searchers use any readily available individuals as participants. 

convergent interviewing an IDI technique for interviewing a 
limited number of experts as participants in a sequential se- 
ries of IDIs; after each successive interview, the researcher 
refines the questions, hoping to converge on the central issues 
in a topic area; sometimes called convergent and divergent 
interviewing. 

correlation the relationship by which two or more variables 
change together, such that systematic changes in one accom- 
pany systematic changes in the other. 

correlational hypothesis a statement indicating that variables 
occur together in some specified manner without implying 
that one causes the other. 

Cramer’s V a measure of association for nominal, nonparamet- 
ric variables; used with larger than 2 X 2 chi-square tables; 
does not provide direction of the association or reflect causa- 
tion; ranges from zero to +1.0. 

creativity session qualitative technique in which an individual 
activity exercise is followed by a sharing/discussion session, 
in which participants build on one another’s creative ideas; 
often used with children; may be conducted before or during 
IDIs or group interviews; usually consists of drawing, visual 
compilation, or writing exercises. 

criterion-related validity see validity, criterion-related. 

criterion variable see dependent variable. 

critical incident technique an IDI technique involving sequen- 
tially asked questions to reveal, in narrative form, what led 
up to an incident being studied; exactly what the observed 
party did or did not do that was especially effective or inef- 
fective; the outcome or result of this action; and why this ac- 
tion was effective or what more effective action might have 
been expected. 

critical path method (CPM) a scheduling tool for complex or 
large research proposals that cites milestones and time in- 
volved between milestones. 

critical value the dividing point(s) between the region of ac- 
ceptance and the region of rejection; these values can be com- 
puted in terms of the standardized random variable due to the 
normal distribution of sample means. 

cross-sectional study the study is conducted only once and re- 
veals a snapshot of one point in time. 

cross-tabulation a technique for comparing data from two or 
more categorical variables. 

cultural interview an IDI technique that asks a participant to 
relate his or her experiences with a culture or subculture, in- 
cluding the knowledge passed on by prior generations and 
the knowledge participants have or plan to pass on to future 
generations. 

cumulative scale a scale development technique in which scale 
items are tested based on a scoring system, and agreement 
with one extreme scale item results also in endorsement of all 
other items that take a less extreme position. 

custom-designed measurement questions measurement ques- 
tions formulated specifically for a particular research project. 

custom researcher crafts a research design unique to the deci- 
sion maker’s dilemma. 
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data information (attitudes, behavior, motivations, attributes, 
etc.) collected from participants or observations (mechanical 
or direct) or from secondary sources. 

data analysis the process of editing and reducing accumulated 
data to a manageable size, developing summaries, looking for 
patterns, and applying statistical techniques. 

data case see record. 

data entry the process of converting information gathered by 
secondary or primary methods to a medium for viewing 
and manipulation; usually done by keyboarding or optical 
scanning. 

data field a single element of data from all participants in a 
study. 

data file a set of data records (all responses from all participants 
in a study). 

data mart intermediate storage facility that compiles locally 
required information. 

data mining applying mathematical models to extract mean- 
ingful knowledge from volumes of data contained within in- 
ternal data marts or data warehouses; purpose is to identify 
valid, novel, useful, and ultimately understandable patterns 
in data. 

data preparation the processes that ensure the accuracy of data 
and their conversion from raw form into categories appropri- 
ate for analysis; includes editing, coding, and data entry. 

data visualization the process of viewing aggregate data on 
multiple dimensions to gain a deeper, intuitive understanding 
of the data. 

data warehouse electronic storehouse where vast arrays of col- 
lected integrated data are stored by categories to facilitate re- 
trieval, interpretation, and sorting by data-mining techniques. 

database a collection of data organized for computerized re- 
trieval; defines data fields, data records, and data files. 

debriefing explains the truth to participants and describes the 
major goals of the research study and the reasons for using 
deception. 

deception occurs when participants are told only part of the 
truth or the truth is fully compromised to prevent biasing par- 
ticipants or to protect sponsor confidentiality. 

decision rule the criterion for judging the attractiveness of two 
or more alternatives when using a decision variable. 

decision support system (DSS) numerous elements of data or- 
ganized for retrieval and use in decision making. 

decision variable a quantifiable characteristic, attribute, or out- 
come on which a choice decision will be made. 

deduction a form of reasoning in which the conclusion must 
necessarily follow from the reasons given; a deduction is 
valid if it is impossible for the conclusion to be false if the 
premises are true. 

demonstration presentation support technique using a visual 
presentation aid to show how something works. 

dependency techniques those techniques in which criterion or 
dependent variables and predictor or independent variables 
are present (e.g., multiple regression, MANOVA, discrimi- 
nant analysis). 

dependent variable (DV) the variable measured, predicted, or 
otherwise monitored by the researcher; expected to be af- 
fected by a manipulation of the independent variable; a.k.a. 
criterion variable. 
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descriptive hypothesis states the existence, size, form, or dis- 
tribution of some variable. 

descriptive statistics display characteristics of the location, 
spread, and shape of a data array. 

descriptive study attempts to describe or define a subject, 
often by creating a profile of a group of problems, people, 
or events, through the collection of data and the tabulation 
of the frequencies on research variables or their interaction; 
the study reveals who, what, when, where, or how much; the 
study concerns a univariate question or hypothesis in which 
the research asks about or states something about the size, 
form, distribution, or existence of a variable. 

deviation scores displays distance of an observation from the 
mean. 

dichotomous question a measurement question that offers two 
mutually exclusive and exhaustive alternatives (nominal data). 

dictionary secondary source that defines words, terms, or jargon 
unique to a discipline; may include information on people, 
events, or organizations that shape the discipline; an excellent 
source of acronyms. 

direct observation occurs when the observer is physically pres- 
ent and personally monitors and records the behavior of the 
participant. 

directory a reference source used to identify contact informa- 
tion (e.g., name, address, phone); many are free, but the most 
comprehensive are proprietary. 

discordant when a subject that ranks higher on one ordinal vari- 
able ranks lower on another variable, the pairs of variables 
are discordant; as discordant pairs increase over concordant 
pairs, the association becomes negative. 

discriminant analysis a technique using two or more indepen- 
dent interval or ratio variables to classify the observations in 
the categories of a nominal dependent variable. 

discussion guide the list of topics to be discussed in an unstruc- 
tured interview (e.g., focus group); a.k.a. interview guide. 

disguised question a measurement question designed to con- 
ceal the question’s and study’s true purpose. 

disk-by-mail (DBM) survey a type of computer-assisted self- 
interview, in which the survey and its management software, 
on computer disk, are delivered by mail to the participant. 

disproportionate sampling see stratified sampling, dispro- 
portionate. 

distribution (of data) the array of value counts from lowest to 
highest value, resulting from the tabulation of incidence for 
each variable by value. 

“don’t know” (DK) response a response given when a partici- 
pant has insufficient knowledge, direction, or willingness to 
answer a question. 

double-barreled question a measurement question that includes 
two or more questions in one that the participant might need to 
answer differently; a question that requests so much content 
that it would be better if separate questions were asked. 

double blind study design in which neither the researcher nor 
the participant knows when a subject is being exposed to the 
experimental treatment. 

double sampling a procedure for selecting a subsample from a 
sample; a.k.a. sequential sampling or multiphase sampling. 

“dummy” table displays data one expects to secure during data 
analysis; each dummy table is a cross-tabulation between two 
or more variables. 


dummy variable nominal variables converted for use in mul- 
tivariate statistics; coded 0, 1, as all other variables must be 
interval or ratio measures. 

dyad (paired interview) a group interview done in pairs (e.g., 
best friends, spouses, superior-subordinate, strangers); used 
often with children. 


EDA see exploratory data analysis. 

editing a process for detecting errors and data omissions and 
correcting them when possible; certifies that minimum data 
quality standards are met. 

eigenvalue proportion of total variance in all the variables that 
is accounted for by a factor. 

electronic test market test that combines store distribution, 
consumer scanner panel data, and household-level media 
delivery. 

empiricism observations and propositions based on sense ex- 
perience and/or derived from such experience by methods of 
inductive logic, including mathematics and statistics. 

encyclopedia a secondary source that provides background or 
historical information on a topic, including names or terms 
that can enhance your search results in other sources. 

enthymeme a truncated syllogism where one or more minor 
premises are left unstated. The presenter gives the primary 
premise and assumes that the audience will supply the miss- 
ing knowledge in order to reach the conclusion. 

environmental control holding constant the physical environ- 
ment of the experiment. 

equal-appearing interval scale an expensive, time-consuming 
type of consensus scaling that results in an interval rating 
scale for attitude measurement; a.k.a. Thurstone scale. 

equivalence when an instrument secures consistent results 
with repeated measures by the same investigator or different 
samples. 

error discrepancy between the sample value and the true popu- 
lation value that occurs when the participant fails to answer 
fully and accurately—either by choice or because of inaccu- 
rate or incomplete knowledge. 

error of central tendency see central tendency (error of). 

error of leniency see leniency (error of). 

error term the deviations of the actual values of Y from the 
regression line (representing the mean value of Y for a par- 
ticular value of X). 

ethics norms or standards of behavior that guide moral choices 
about research behavior. 

ethnographic research see ethnography. 

ethnography interviewer and participant collaborate in a field- 
setting participant observation and unstructured interview; 
typically takes place where the behavior being observed oc- 
curs (e.g., participant’s home). 

ethos how well the audience believes that the presenter is quali- 
fied to speak on the particular subject; determined by the 
perception of a presenter’s character, his or her past experi- 
ence, or the credibility and experience of those the presenter 
evokes. 

event sampling the process of selecting some elements or 
behavioral acts or conditions from a population of observ- 
able behaviors or conditions to represent the population as 
a whole. 


ex post facto design after-the-fact report on what happened to 
the measured variable. 

example a true or hypothetical instance used to a clarify com- 
plex idea. 

executive summary (final report) this document is written as the 
last element of a research report and either is a concise summary 
of the major findings, conclusions, and recommendations or is 
a report in miniature, covering all aspects in abbreviated form. 

executive summary (proposal) an informative abstract provid- 
ing the essentials of the proposal without the details. 

experience survey (expert interview) semistructured or un- 
structured interviews with experts on a topic or dimension of 
a topic; an exploratory technique in which knowledgeable ex- 
perts share their ideas about important issues or aspects of the 
subject and relate what is important across the subject’s range 
of experience; usually involves a personal or phone interview. 

experiment (experimental study) study involving intervention 
(manipulation of one or more variables) by the researcher be- 
yond that required for measurement to determine the effect 
on another variable. 

experimental treatment the manipulated independent variable. 

expert group interview group interview consisting of individu- 
als exceptionally knowledgeable about the issues or topics to 
be discussed. 

expert interview a discussion with someone knowledgeable 
about the problem or its possible solutions. 

expert opinion (testimony) opinions of recognized experts 
who possess credibility for your audience on a topic; used as 
support or proof. 

explanatory (causal) hypothesis a statement that describes 
a relationship between two variables in which one variable 
leads to a specified effect on the other variable. 

explanatory study attempts to explain an event, act, or charac- 
teristic measured by research. 

explicit attitude an expressed positive or negative evaluation. 

exploration the process of collecting information to formulate 
or refine management, research, investigative, or measure- 
ment questions; loosely structured studies that discover future 
research tasks, including developing concepts, establishing 
priorities, developing operational definitions, and improving re- 
search design; a phase of a research project where the researcher 
expands understanding of the management dilemma, looks for 
ways others have addressed and/or solved problems similar to 
the management dilemma or management question, and gath- 
ers background information on the topic to refine the research 
question; a.k.a. exploratory study or exploratory research. 

exploratory data analysis (EDA) patterns in the collected data 
guide the data analysis or suggest revisions to the preliminary 
data analysis plan. 

exploratory research see exploration. 

exploratory study sce exploration. 

exposition statement that describes without attempting to explain. 

extemporaneous presentation an audience-centered, pre- 
planned speech made from minimal notes; generates a 
presentation that is natural, conversational, and flexible to 
audience interests. 

external validity occurs when an observed causal relationship 
can be generalized across persons, settings, and times. 

extralinguistic behavior the vocal, temporal, interactive, and 
verbal stylistic behaviors of human participants. 
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extraneous variable (EV) variable to assume (because it has 
little effect or its impact is randomized) or exclude from a 
research study. 

extranet a private network that uses the Internet protocols and 
the public telecommunication system to share a business’s 
information, data, or operations with external suppliers, ven- 
dors, or customers. 

eye contact a meeting of the eyes between two people that ex- 
presses meaningful nonverbal communication revealing con- 
cern, warmth, and authenticity. 


F ratio F test statistic comparing measurements of k indepen- 
dent samples. 

fact a piece of information about a situation that exists or an 
event known to have occurred; it takes the form of a state- 
ment about verifiable data that support the presenter’s 
argument. 

factor denotes an independent variable (IV) in an experiment; 
factors are divided into treatment levels for the experiment. 

factor analysis a technique for discovering patterns among 
the variables to determine if an underlying combination of 
the original variables (a factor) can summarize the original 
set. 

factor scales types of scales that deal with multidimensional 
content and underlying dimensions, such as scalogram, fac- 
tor, and cluster analyses, and metric and nonmetric multidi- 
mensional scaling. 

factors in factor analysis, the result of transforming a set of 
variables into a new set of composite variables; these factors 
are linear and not correlated with each other. 

field conditions the actual environmental conditions in which 
the dependent variable occurs. 

field experiment a study of the dependent variable in actual en- 
vironmental conditions. 

filter question see screen question. 

findings nondisclosure a type of confidentiality; the sponsor 
restricts the researcher from discussing the findings of the 
research project. 

five-number summary the median, the upper and lower quar- 
tiles, and the largest and smallest observations of a variable’s 
distribution. 

fixed-sum scale see constant-sum scale. 

flow aids a design principle that provides a visual aid that re- 
veals to the audience where the presenter is within the overall 
presentation. 

focus group the simultaneous involvement of a small number 
of research participants (usually 8 to 10) who interact at the 
direction of a moderator in order to generate data on a par- 
ticular issue or topic; widely used in exploratory studies; usu- 
ally lasts 90 minutes to two hours; can be conducted in person 
or via phone or videoconference. 

forced-choice rating scale requires that participants select 
from available alternatives. 

forced ranking scale a scale in which the participant orders 
several objects or properties of objects; faster than paired 
comparison to obtain a rank order. 

formal study research question—driven process involving pre- 
cise procedures for data collection and interpretation; tests 
the hypothesis or answers the research questions posed. 
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format how the information is presented and how easy it is to find 
a specific piece of information within a secondary source; one 
of five factors used to evaluate the value of a secondary source. 

forward selection in modeling and regression, sequentially 
adds the variables to a regression model that results in the 
largest R? increase; see also backward elimination and step- 
wise selection. 

free-response question a measurement question in which the 
participant chooses the words to frame the answer; a.k.a. 
open ended question (nominal, ordinal, or ratio data). 

frequency distribution ordered array of all values for a variable. 

frequency table arrays category codes from lowest value to 
highest value, with columns for count, percent, valid percent, 
and cumulative percent. 

full-service researchers a firm with both quantitative and 
qualitative methodology expertise that conducts all phases of 
research from planning to insight development, often serving 
as both research firm and consultant. 

funnel approach a type of question sequencing that moves the 
participant from general to more specific questions and is de- 
signed to learn the participant’s frame of reference while ex- 
tracting full disclosure of information on the topic (nominal, 
ordinal, interval, or ratio data). 


gamma (y) uses a preponderance of evidence of concordant 
pairs versus discordant pairs to predict association; the 
gamma value is the proportional reduction of error when 
prediction is done using preponderance of evidence (values 
from —1.0 to +1.0). 

geographic chart uses a map to show regional variations in data. 

gestures a form of nonverbal communication made with a part 
of the body, used instead of or in combination with verbal 
communication; allows presenters to express a variety of 
feelings and thoughts, positive and negative. 

goodness of fit a measure of how well the regression model is 
able to predict Y. 

graphic rating scale a scale in which the participant places his 
or her response along a line or continuum; the score or mea- 
surement is its distance in millimeters from either endpoint. 

grounded theory an IDI technique in which analysis of the data 
takes place simultaneously with its collection, with the pur- 
pose of developing general concepts or theories with which 
to analyze the data. 

group interview a data collection method using a single in- 
terviewer who simultaneously interviews more than one re- 
search participant. 


halo effect error caused when prior observations influence per- 
ceptions of current observations. 

handbook a secondary source used to identify key terms, peo- 
ple, or events relevant to the management dilemma or man- 
agement question. 

heterogeneous group participant group consisting of individu- 
als with a variety of opinions, backgrounds, and actions rela- 
tive to a topic. 

histogram a graphical bar chart that groups continuous data 
values into equal intervals with one bar for each interval; es- 
pecially useful for revealing skewness, kurtosis, and modal 
pattern. 


holdout sample the portion of the sample (usually “4 or 4) ex- 
cluded for later validity testing when the estimating question 
is first computed; the equation is then used on the holdout 
data to calculate R? for comparison. 

homogeneous group participant group consisting of individu- 
als with similar opinions, backgrounds, and actions relative 
to a topic. 

hypothesis a proposition formulated for empirical testing; a 
tentative descriptive statement that describes the relationship 
between two or more variables. 

hypothetical construct construct inferred only from data; its 
presumption must be tested. 


ill-defined problem one that addresses complex issues and can- 
not be expressed easily, concisely, or completely. 

imaginary universe a projective technique (imagination exer- 
cise) in which participants are asked to assume that the brand 
and its users populate an entire universe; they then describe 
the features of this new world. 

imagination exercises a projective technique in which partici- 
pants are asked to relate the properties of one thing/person/ 
brand to another. 

implicit attitude an attitude about one object that influences the 
attitude about other objects. 

impromptu speaking a speech that does not involve prepara- 
tion, and evolves spontaneously in response to some stimu- 
lus, such as a question. 

incidence the number of elements in the population belonging 
to the category of interest, divided by the total number of ele- 
ments in the population. 

independent variable (IV) the variable manipulated by the re- 
searcher, thereby causing an effect or change on the depen- 
dent variable. 

index secondary data source that helps identify and locate a sin- 
gle book, journal article, author, etc., from among a large set. 

indirect observation occurs when the recording of data is done 
by mechanical, photographic, or electronic means. 

individual depth interview (IDI) a type of interview that en- 
courages the participant to talk extensively, sharing as much 
information as possible; usually lasts one or more hours; 
three types: structured, semistructured, and unstructured. 

induction (inductive reasoning) to draw a conclusion from 
one or more particular facts or pieces of evidence; the con- 
clusion explains the facts. 

inferential statistics includes the estimation of population val- 
ues and the testing of statistical hypotheses. 

informed consent participant gives full consent to participation 
after receiving full disclosure of the procedures of the pro- 
posed survey. 

interaction effect the influence that one factor has on another factor. 

intercept (f,) one of two regression coefficients; the value for 
the linear function when it crosses the Y axis or the estimate 
of Y when X is zero. 

intercept interview a face-to-face communication that targets 
participants in a centralized location. 

interdependency techniques techniques in which criterion or 
dependent variables and predictor or independent variables 
are not present (e.g., factor analysis, cluster analysis, multi- 
dimensional scaling). 


internal consistency characteristic of an instrument in which 
the items are homogeneous; measure of reliability. 

internal database collection of data stored by an organization. 

internal validity the ability of a research instrument to measure 
what it is purported to measure; occurs when the conclusion(s) 
drawn about a demonstrated experimental relationship truly 
implies cause. 

interquartile range (IQR) measures the distance between 
the first and third quartiles of a data distribution; a.k.a. mid- 
spread, the distance between the hinges in a boxplot. 

interval estimate range of values within which the true popula- 
tion parameter is expected to fall. 

interval scale scale with the properties of order and equal dis- 
tance between points and with mutually exclusive and ex- 
haustive categories; data that incorporate equality of interval 
(the distance between one measure and the next measure); 
e.g., temperature scale. 

intervening variable (IVV) a factor that affects the observed 
phenomenon but cannot be seen, measured, or manipulated; 
thus its effect must be inferred from the effects of the inde- 
pendent and moderating variables on the dependent variable. 

interview phone, in-person, or videoconference communica- 
tion approach to collecting data. 

interview guide see discussion guide. 

interview schedule question list used to guide a structured in- 
terview; a.k.a. questionnaire. 

interviewer error error that results from interviewer influ- 
ence of the participant; includes problems with motivation, 
instructions, voice inflections, body language, question or 
response order, or cheating via falsification of one or more 
responses. 

intranet a private network that is contained within an enter- 
prise; access is restricted to authorized audiences; usually 
behind a security firewall. 

investigative questions questions the researcher must answer 
to satisfactorily answer the research question; what the man- 
ager feels he or she needs to know to arrive at a conclusion 
about the management dilemma. 

item analysis scale development in which instrument design- 
ers develop instrument items and test them with a group of 
participants to determine which highly discriminate between 
high and low raters. 


jargon language unique to a profession or discipline; when un- 
known by the audience can reduce the clarity of the message. 

judgment sampling a purposive sampling in which the re- 
searcher arbitrarily selects sample units to conform to some 
criterion. 


k-independent-samples tests significance tests in which mea- 
surements are taken from three or more samples (ANOVA 
for interval or ratio measures, Kruskal-Wallis for ordinal 
measures, chi-square for nominal measures). 

k-related-samples tests compares measurements from more 
than two groups from the same sample or more than two 
measures from the same subject or participant (ANOVA for 
interval or ratio measures, Friedman for ordinal measures, 
Cochran Q for nominal measures). 
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kinesics the study of the use of body motion communication. 

kinesthetic learners people who learn by doing, moving, and 
touching. 

kurtosis measure of a data distribution’s peakedness or flatness 
(ku); a neutral distribution has a ku of 0, a flat distribution is 
negative, and a peaked distribution is positive. 


laboratory conditions studies that occur under conditions that 
do not simulate actual environmental conditions. 

laddering (benefit chain) a projective technique in which par- 
ticipants are asked to link functional features to their physical 
and psychological benefits, both real and ideal. 

lambda (A) a measure of how well the frequencies of one nomi- 
nal variable predict the frequencies of another variable; val- 
ues (vary between zero and 1.0) show the direction of the 
association. 

leading question a measurement question whose wording sug- 
gests to the participant the desired answer (nominal, ordinal, 
interval, or ratio data). 

leniency (error of) a participant, within a series of evaluations, 
consistently expresses judgments at one end of a scale; an 
error that results when the participant is consistently an easy 
rater. 

letter of transmittal the element of the final report that pro- 
vides the purpose of, scope of, authorization for, and limita- 
tions of the study; not necessary for internal projects. 

level of significance the probability of rejecting a true null 
hypothesis. 

life history an IDI technique that extracts from a single partici- 
pant memories and experiences from childhood to the present 
day regarding a product or service category, brand, or firm. 

Likert scale a variation of the summated rating scale, this scale 
asks a rater to agree or disagree with statements that express 
either favorable or unfavorable attitudes toward the object. 
The strength of attitude is reflected in the assigned score, 
and individual scores may be totaled for an overall attitude 
measure. 

limiters database search protocol for narrowing a search; com- 
monly include date, publication type, and language. 

line graph a statistical presentation technique used for time se- 
ries and frequency distributions over time. 

linearity an assumption of correlation analysis, that the col- 
lection of data can be described by a straight line passing 
through the data array. 

linguistic behavior the human verbal behavior during conver- 
sation, presentation, or interaction. 

literature review recent or historically significant research 
studies, company data, or industry reports that act as the basis 
for the proposed study. 

literature search a review of books, articles in journals or pro- 
fessional literature, research studies, and Web-published ma- 
terials that relate to the management dilemma, management 
question, or research question. 

loadings in principal components analysis, the correlation coef- 
ficients that estimate the strength of the variables that com- 
pose the factor. 

logos the logical argument; requires supporting evidence and 
analytical techniques that reveal and uphold the researcher’s 
findings and conclusions. 
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longitudinal study the study includes repeated measures over 
an extended period of time, tracking changes in variables 
over time; includes panels or cohort groups. 


mail survey a relatively low-cost self-administered study both 
delivered and returned via mail. 

main effect the average direct influence that a particular treat- 
ment of the IV has on the DV independent of other factors. 

management dilemma the problem or opportunity that requires 
a decision; a symptom of a problem or an early indication of 
an opportunity. 

management question the management dilemma restated in 
question format; categorized as “‘choice of objectives,” “gen- 
eration and evaluation of solutions,” or “troubleshooting or 
control of a situation.” 

management report a report written for the nontechnically ori- 
ented manager or client. 

management-research question hierarchy process of sequen- 
tial question formulation that leads a manager or researcher 
from management dilemma to measurement questions. 

manuscript reading the verbatim reading of a fully written 
presentation. 

mapping rules a scheme for assigning numbers to aspects of an 
empirical event. 

marginal(s) a term for the column and row totals in a 
cross-tabulation. 

matching a process analogous to quota sampling for assigning 
participants to experimental and control groups by having 
participants match every descriptive characteristic used in the 
research; used when random assignment is not possible; an 
attempt to eliminate the effect of confounding variables that 
group participants so that the confounding variable is present 
proportionally in each group. 

MDS see multidimensional scaling. 

mean the arithmetic average of a data distribution. 

mean square the variance computed as an average or mean. 

measurement assigning numbers to empirical events in com- 
pliance with a mapping rule. 

measurement questions the questions asked of the participants 
or the observations that must be recorded. 

measures of location term for measure of central tendency in a 
distribution of data; see also central tendency. 

measures of shape statistics that describe departures from the sym- 
metry of a distribution; a.k.a. moments, skewness, and kurtosis. 

measures of spread statistics that describe how scores cluster 
or scatter in a distribution; a.k.a. dispersion or variability 
(variance, standard deviation, range, interquartile range, and 
quartile deviation). 

median the midpoint of a data distribution where half the cases 
fall above and half the cases fall below. 

memorization the act of committing to memory all details of a 
presentation. 

metaphor a figure of speech in which an implicit comparison is 
made between two unlike things that actually have something 
important in common. 

metaphor elicitation technique an individual depth interview 
that reveals participants’ hidden or suppressed attitudes and 
perceptions by having them explain collected images and 
each image’s relation to the topic being studied. 


method of least squares a procedure for finding a regression 
line that keeps errors (deviations from actual value to the line 
value) to a minimum. 

metric measures statistical techniques using interval and ratio 
measures. 

mini-group a group interview involving two to six people. 

missing data information that is missing about a participant or 
data record; should be discovered and rectified during data 
preparation phase of analysis; e.g., miscoded data, out-of- 
range data, or extreme values. 

mode the most frequently occurring value in a data distribution; 
data may have more than one mode. 

model a representation of a system that is constructed to study 
some aspect of that system or the system as a whole. 

moderating variable (MV) a second independent variable, be- 
lieved to have a significant contributory or contingent effect 
on the originally stated IV-DV relationship. 

moderator a trained interviewer used for group interviews such 
as focus groups. 

monitoring a classification of data collection that includes ob- 
servation studies and data mining of organizational databases. 

motivated sequence a presentation planning approach that in- 
volves the ordering of ideas to follow the normal processes 
of human thinking; motivates an audience to respond to the 
presenter’s purpose. 

multicollinearity occurs when more than two independent vari- 
ables are highly correlated. 

multidimensional scale a scale that seeks to simultaneously 
measure more than one attribute of the participant or object. 

multidimensional scaling (MDS) a scaling technique to simul- 
taneously measure more than one attribute of the participant 
or object; results are usually mapped; develops a geometric 
picture or map of the locations of some objects relative to 
others on various dimensions or properties; especially useful 
for difficult-to-measure constructs. 

multiphase sampling see double sampling. 

multiple-choice, multiple-response scale a scale that offers 
the participant multiple options and solicits one or more an- 
swers (nominal or ordinal data); a.k.a. checklist. 

multiple-choice question a measurement question that offers 
more than two category responses but seeks a single answer. 

multiple-choice, single-response scale a scale that poses more 
than two category responses but seeks a single answer, or one 
that seeks a single rating from a gradation of preference, in- 
terest, or agreement (nominal or ordinal data); a.k.a. multiple- 
choice question. 

multiple comparison tests compare group means following the 
finding of a statistically significant F test. 

multiple rating list scale a single interval or ordinal numeri- 
cal scale where raters respond to a series of objects; results 
facilitate visualization. 

multiple regression a statistical tool used to develop a 
selfweighting estimating equation that predicts values for 
a dependent variable from the values of independent vari- 
ables; controls confounding variables to better evaluate the 
contribution of other variables; tests and explains a causal 
theory. 

multivariate analysis statistical techniques that focus upon and 
bring out in bold relief the structure of simultaneous relation- 
ships among three or more phenomena. 


multivariate analysis of variance (MANOVA) assesses the 
relationship between two or more dependent variables and 
classificatory variables or factors; frequently used to test dif- 
ferences among related samples. 


narrative see oral history. 

narrative pattern a presentation organizational pattern that in- 
volves the use of stories as the primary vehicle for communi- 
cating the presenter’s message. 

negative leniency (error of) an error that results when the par- 
ticipant is consistently a hard or critical rater. 

nominal scale scale with mutually exclusive and collectively 
exhaustive categories, but without the properties of order, 
distance, or unique origin. 

noncontact rate ratio of noncontacts (no answer, busy, answer- 
ing machine, and disconnects) to all potential contacts. 

nondisclosure various types of confidentiality involving re- 
search projects, including sponsor, findings, and purpose 
nondisclosures. 

nonexpert group participants in a group interview who have 
at least some desired information but at an unknown level. 

nonmetric measures statistical techniques using ordinal and 
nominal measures (nonparametric). 

nonparametric tests significance tests for data derived from 
nominal and ordinal scales. 

nonprobability sampling an arbitrary and subjective procedure 
in which each population element does not have a known 
nonzero chance of being included; no attempt is made to gen- 
erate a statistically representative sample. 

nonresistant statistics a statistical measure that is susceptible to 
the effects of extreme values; e.g., mean, standard deviation. 

nonresponse error error that develops when an interviewer can- 
not locate the person with whom the study requires communi- 
cation or when the targeted participant refuses to participate; 
especially troublesome in studies using probability sampling. 

nonverbal behavior human behaviors not related to conver- 
sation (e.g., body movement, facial expressions, exchanged 
glances, eyeblinks). 

nonverbal communication meaning conveyed through other than 
verbal means; encompasses clothing and bodily characteristics, 
physical environment (physical space and time), movement and 
body position (including kinesics, posture, gesture, touch), eye 
gaze, and paralanguage (nonverbal cues of the voice). 

nonverbal observation observation of human behavior without 
the use of conversation between observers and participants. 

normal distribution a frequency distribution of many natural 
phenomena; graphically shaped like a symmetrical curve. 

normal probability plot compares the observed values with 
those expected from a normal distribution. 

null hypothesis (H,) assumption that no difference exists be- 
tween the sample parameter and the population statistic. 

numerical scale a scale in which equal intervals separate the 
numeric scale points, while verbal anchors serve as labels for 
the extreme points. 


objects concepts defined by ordinary experience. 
observation the full range of monitoring behavioral and nonbe- 
havioral activities and conditions (including record analysis, 
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physical condition analysis, physical process analysis, non- 
verbal analysis, linguistic analysis, extralinguistic analysis, 
and spatial analysis). 

observation checklist a measurement instrument for recording 
data in an observation study; analogous to a questionnaire in 
a communication study. 

observation playgroup an observation technique that involves 
observing children at play, often with targeted objects (toys 
or materials); observers are usually behind one-way mirrors. 

observation study a monitoring approach to collecting data in 
which the researcher inspects the activities of a subject or the 
nature of some material without attempting to elicit responses 
from anyone; a.k.a. monitoring. 

observed significance level the probability value compared to 
the significance level (e.g., .05) chosen for testing and on this 
basis the null hypothesis is either rejected or not rejected. 

observer drift a source of error affecting categorization caused 
by decay in reliability or validity of recorded observations 
over time. 

OCR see optical character recognition. 

omnibus researcher fields research studies, often by survey, at 
regular, predetermined intervals. 

omnibus study combines the questions of several decision 
makers who need information from the same population. 

one-sample tests tests that involve measures taken from a sin- 
gle sample compared to a specified population. 

one-tailed test a test of a null hypothesis that assumes the sam- 
ple parameter is not the same as the population statistic, but 
that the difference is in only one direction. 

online focus group a type of focus group in which participants 
use the technology of the Internet, including e-mail, websites, 
Usenet newsgroups, or an Internet chat room, to approximate 
the interaction of a face-to-face focus group. 

open-ended question see free-response question. 

operational definition a definition for a variable stated in terms 
of specific testing criteria or operations, specifying what must 
be counted, measured, or gathered through our senses. 

operationalized the process of transforming concepts and con- 
structs into measurable variables suitable for testing. 

optical character recognition (OCR) software programs that 
transfer printed text into a computer file in order to edit and 
use the information without rekeying the data. 

optical mark recognition (OMR) software that uses a 
spreadsheet-style interface to read and process data from 
user-created forms. 

optical scanning a data entry process whereby answers are re- 
corded on computer-readable forms and then scanned to form 
a data record; reduces data handling and the errors that ac- 
company such data handling. 

oral history (narrative) an IDI technique that asks participants 
to relate their personal experiences and feelings related to 
historical events or past behavior. 

ordinal measures measures of association between variables 
generating ordinal data. 

ordinal scale scale with mutually exclusive and collectively ex- 
haustive categories, as well as the property of order, but not 
distance or unique origin; data capable of determining greater- 
than, equal-to, or less-than status of a property or an object. 

outliers data points that exceed +1.5 the interquartile range 


(IQR). 


662 >glossary 


p value probability of observing a sample value as extreme as, 
or more extreme than, the value actually observed, given that 
the null hypothesis is true. 

pace the rate at which the printed page presents information to 
the reader; it should be slower when the material is complex, 
faster when the material is straightforward. 

paired-comparison scale the participant chooses a preferred 
object between several pairs of objects on some property; re- 
sults in a rank ordering of objects. 

paired interview see dyad. 

panel a group of potential participants who have indicated a 
willingness to participate in research studies; often used for 
longitudinal communication studies; may be used for both 
qualitative and quantitative research. 

paralanguage nonverbal communication that includes such 
vocal elements as tone, pitch, rhythm, pause, timbre, loud- 
ness, and inflection. 

parametric tests significance tests for data from interval and 
ratio scales. 

Pareto diagram a graphical presentation that represents fre- 
quency data as a bar chart, ordered from most to least, over- 
laid with a line graph denoting the cumulative percentage at 
each variable level. 

participant the subject, respondent, or sample element in a 
research study. 

participant-initiated response error error that occurs when 
the participant fails to answer fully and accurately—either by 
choice or because of inaccurate or incomplete knowledge. 

participant observation when the observer is physically in- 
volved in the research situation and interacts with the partici- 
pant to influence some observation measures. 

participants’ perceptual awareness the subtle or major 
changes that occur in participants’ responses when they per- 
ceive that a research study is being conducted. 

path analysis describes through regression an entire structure 
of linkages that have been advanced by a causal theory. 

path diagram presents predictive and associative relationships 
among constructs and indicators in a structural model. 

pathos an appeal to an audience’s sense of identity, self-interest, 
and emotions, which relies on an emotional connection be- 
tween the presenter and his or her audience. 

Pearson correlation coefficient the r symbolizes the estimate 
of strength of linear association and its direction between in- 
terval and ratio variables; based on sampling data and varies 
over a range of +1 to —1; the prefix (+, —) indicates the 
direction of the relationship (positive or inverse), while the 
number represents the strength of the relationship (the closer 
to 1, the stronger the relationship; 0 = no relationship); and 
the p represents the population correlation. 

performance anxiety (stage fright) a fear produced by the 
need to make a presentation in front of an audience or before 
a camera. 

permission surveying the act of surveying prospects or cus- 
tomers who have given permission for such engagement, 
usually through panel membership. 

personification a projective technique (imagination exercise) 
in which participants are asked to imagine inanimate objects 
with the traits, characteristics and features, and personalities 
of humans. 


phi (d) a measure of association for nominal, nonparametric 
variables; ranges from zero to +1.0 and is used best with 
2 X 2 chi-square tables; does not provide direction of the 
association or reflect causation. 

photographic framing the practice of creating a focal point 
for all visuals used in presentations. 

physical condition analysis the recording of observations of 
current conditions resulting from prior decisions; includes 
inventory, signs, obstacles or hazards, cleanliness, etc. 

physical trace a type of observation that collects measures of 
wear data (erosion) and accretion data (deposit) rather than 
direct observation (e.g., a study of trash). 

pictograph a bar chart using pictorial symbols rather than bars 
to represent frequency data; the symbol has an association 
with the subject of the statistical presentation and one symbol 
unit represents a specific count of that variable. 

pie chart uses sections of a circle (slices of a pie) to represent 
100 percent of a frequency distribution of the subject being 
graphed; not appropriate for changes over time. 

pilot test a trial collection of data to detect weaknesses in design 
and instrumentation and provide proxy data for selection of a 
probability sample; see also pretesting. 

point estimate sample mean; our best predictor of the unknown 
population mean. 

population the elements about which we wish to make some 
inferences. 

population element the individual participant or object on 
which the measurement is taken; a.k.a. population unit, sam- 
ple element, sample unit. 

population parameter a summary descriptor of a variable of 
interest in the population; e.g., incidence, mean, variance. 

population proportion of incidence the number of category el- 
ements in the population, divided by the number of elements 
in the population. 

portal a Web page that serves as a gateway to more remote 
Web publications; usually includes one or more directories, 
search engines, and other user features such as news and 
weather. 

posture and body orientation communication of nonverbal 
messages by the way you walk and stand. 

power of the test 1 minus the probability of committing a Type 
II error (1 minus the probability that we will correctly reject 
the false null hypothesis). 

practical significance when a statistically significant difference 
has real importance to the decision maker. 

practicality a characteristic of sound measurement concerned 
with a wide range of factors of economy, convenience, and 
interpretability. 

PRE see proportional reduction in error. 

precision one of the considerations in determining sample va- 
lidity: the degree to which estimates from the sample reflect 
the measure taken by a census; measured by the standard 
error of the estimate—the smaller the error, the greater the 
precision of the estimate. 

precoding assigning codebook codes to variables in a study and 
recording them on the questionnaire; eliminates a separate 
coding sheet. 

predesigned measurement questions questions that have been 
formulated and tested by previous researchers, are recorded 


in the literature, and may be applied literally or be adapted for 
the project at hand. 

prediction and confidence bands bow-tie-shaped confidence 
intervals around a predictor; predictors farther from the mean 
have larger bandwidths in regression analysis. 

predictive study is used to determine if a relationship exists be- 
tween two or more variables. When causation is established, 
one variable can be used to predict the other. In business re- 
search, studies conducted to evaluate specific courses of ac- 
tion or to forecast current or future values. 

predictor variable see independent variable. 

pretasking a variety of creative and mental exercises to prepare 
participants for individual or group interviews, such as an IDI 
or focus group; intended to increase understanding of partici- 
pants’ own thought processes and bring their ideas, opinions, 
and attitudes to the surface. 

pretesting the assessment of questions and instruments before 
the start of a study; an established practice for discovering 
errors in questions, question sequencing, instructions, skip 
directions, etc.; see also pilot test. 

primacy effect order bias in which the participant tends to 
choose the first alternative; a principle affecting presentation 
organization in which the first item in a list is initially dis- 
tinguished as important and may be transferred to long-term 
memory; implies an important argument should be first in 
your presentation. 

primary data data the researcher collects to address the specific 
problem at hand—the research question. 

primary sources original works of research or raw data with- 
out interpretation or pronouncements that represent an of- 
ficial opinion or position; include memos, letters, complete 
interviews or speeches, laws, regulations, court decisions, 
and most government data, including census, economic, and 
labor data; the most authoritative of all sources. 

principal components analysis one method of factor analysis 
that transforms a set of variables into a new set of composite 
variables; these variables are linear and not correlated with 
each other; see also factor analysis. 

principle of appropriate knowledge only information compat- 
ible with your audience’s knowledge level should be used in 
a presentation. 

principle of capacity limitations the audience cannot process 
large amounts of information at one time. 

principle of disciminability two properties must differ by 
a large amount for the difference to be discerned by your 
audience. 

principle of informative changes your audience will expect 
anything you speak about, demonstrate, or show in your pre- 
sentation to convey important information. 

principle of perceptual organization your audience will auto- 
matically group items together, even if you don’t give them 
such groupings; this facilitates their absorbing and storing 
large amounts of information. 

principle of relevance only information critical to understand- 
ing should be presented. 

principle of salience your audience’s attention is drawn to 
large perceptible differences. 

probability sampling a controlled, randomized procedure that 
ensures that each population element is given a known non- 
zero chance of selection; used to draw participants that are 
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representative of a target population; necessary for projecting 
findings from the sample to the target population. 

probing techniques for stimulating participants to answer more 
fully and relevantly to posed questions. 

process (activity) analysis observation by a time study of stages 
in a process, evaluated on both effectiveness and efficiency; 
includes traffic flow within distribution centers and retailers, 
paperwork flow, customer complaint resolution, etc. 

project management the process of planning and managing 
a detailed project, through tables and charts with detail re- 
sponsibilities and deadlines; details relationship between 
researchers, their assistants, sponsors, and suppliers; often 
results in a Gantt chart. 

projective techniques qualitative methods that encourage the 
participant to reveal hidden or suppressed attitudes, ideas, 
emotions, and motives; various techniques (e.g., sentence 
completion tests, cartoon or balloon tests, word association 
tests) used as part of an interview to disguise the study objec- 
tive and allow the participant to transfer or project attitudes 
and behavior on sensitive subjects to third parties; the data 
collected via these techniques are often difficult to interpret 
(nominal, ordinal, or ratio data). 

properties characteristics of objects that are measured; a per- 
son’s properties are his or her weight, height, posture, hair 
color, etc. 

proportion percentage of elements in the distribution that meet 
a criterion. 

proportional reduction in error (PRE) measures of associa- 
tion used with contingency tables (a.k.a cross-tabulations) to 
predict frequencies. 

proportionate sampling see stratified sampling, propor- 
tionate. 

proposal a work plan, prospectus, outline, statement of in- 
tent, or draft plan for a research project, including proposed 
budget. 

proposition a statement about concepts that may be judged as 
true or false if it refers to observable phenomena. 

proprietary methodology a research program or technique that 
is owned by a single firm; may be branded. 

proxemics the study of the use of space; the study of how peo- 
ple organize the territory around them and the discrete dis- 
tances they maintain between themselves and others. 

proximity an index of perceived similarity or dissimilarity be- 
tween objects. 

pure research (basic research) designed to solve problems of 
a theoretical nature with little direct impact on strategic or 
tactical decisions. 

purpose the explicit or hidden agenda of the authors of the 
secondary source; one of five factors in secondary source 
evaluation. 

purpose nondisclosure a type of confidentiality; occurs when 
the sponsor camouflages the true research objective of the 
research project. 

purposive sampling anonprobability sampling process in which 
researchers choose participants arbitrarily for their unique 
characteristics or their experiences, attitudes, or perceptions. 


Q-sort participant sorts a deck of cards (representing properties 
or objects) into piles that represent points along a continuum. 
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qualitative research interpretive techniques that seek to de- 
scribe, decode, translate, and otherwise come to terms with 
the meaning, not the frequency, of certain phenomena; a 
fundamental approach of exploration, including individual 
depth interviews, group interviews, participant observation, 
videotaping of participants, projective techniques and psy- 
chological testing, case studies, street ethnography, elite in- 
terviewing, document analysis, and proxemics and kinesics; 
see also content analysis. 

qualitative techniques nonquantitative data collection used to 
increase understanding of a topic. 

quantitative research the precise count of some behavior, 
knowledge, opinion, or attitude. 

quartile deviation (Q) a measure of dispersion for ordinal data 
involving the median and quartiles; the median plus one 
quartile deviation on either side encompasses 50 percent of 
the observations and eight cover the full range of data. 

questionnaire an instrument delivered to the participant via per- 
sonal (intercept, phone) or nonpersonal (computer-delivered, 
mail-delivered) means that is completed by the participant. 

quota matrix a means of visualizing the matching process. 

quota sampling purposive sampling in which relevant charac- 
teristics are used to stratify the sample. 


random assignment a process that uses a randomized sample 
frame for assigning sample units to test groups in an attempt 
to ensure that the groups are as comparable as possible with 
respect to the DV; each subject must have an equal chance for 
exposure to each level of the independent variable. 

random dialing a computerized process that chooses phone ex- 
changes or exchange blocks and generates numbers within 
these blocks for telephone surveys. 

random error error that occurs erratically, without pattern; see 
also sampling error. 

randomization using random selection procedures to assign 
sample units to either the experimental or control group to 
achieve equivalence between groups. 

range the difference between the largest and smallest scores in 
the data distribution; a very rough measure of spread of a 
dispersion. 

range tests see multiple comparison tests. 

ranking question a measurement question that asks the partici- 
pant to compare and order two or more objects or properties 
using a numeric scale. 

ranking scale a scale that scores an object or property by mak- 
ing a comparison and determining order among two or more 
objects or properties; uses a numeric scale and provides ordi- 
nal data; see also ranking question. 

rating question a question that asks the participant to posi- 
tion each property or object on a verbal, numeric, or graphic 
continuum. 

rating scale a scale that scores an object or property without 
making a direct comparison to another object or property; 
either verbal, numeric, or graphic; see also rating question. 

ratio scale a scale with the properties of categorization, order, 
equal intervals, and unique origin; numbers used as measure- 
ments have numeric value; e.g., weight of an object. 

reactivity response the phenomenon that occurs when partici- 
pants alter their behavior due to the presence of the observer. 


readability index measures the difficulty level of written mate- 
rial; e.g., Flesch Reading Ease Score, Flesch Kincaid Grade 
Level, Gunning’s Fog Index; most word processing programs 
calculate one or several of the indexes. 

recency effect order bias occurs when the participant tends to 
choose the last alternative; in presentations, people remember 
what they hear at the end of the list of arguments in a speech, 
recalling those items best; implies an important argument 
should be the last in your presentation. 

reciprocal relationship occurs when two variables mutually 
influence or reinforce each other. 

record a set of data fields that are related, usually by subject or 
participant; represented by rows in a spreadsheet or statistical 
database; a.k.a. data case, data record. 

record analysis the extraction of data from current or historical 
records, either private or in the public domain; a technique 
of data mining. 

recruitment screener semistructured or structured interview 
guide designed to ensure the interviewer that the prospect 
will be a good participant for the planned research. 

refusal rate ratio of participants who decline the interview to 
all potential/eligible contacts. 

region of acceptance area between the two regions of rejection 
based on a chosen level of significance (two-tailed test) or 
the area above/below the region of rejection (one-tailed test). 

region of rejection area beyond the region of acceptance set by 
the level of significance. 

regression analysis uses simple and multiple predictions to pre- 
dict Y from X values. 

regression coefficients intercept and slope coefficients; the two 
association measures between X and Y variables. 

relational hypothesis describes the relationship between two 
variables with respect to some case; relationships are correla- 
tional or explanatory. 

relationship a design principle that encourages the use of visual 
techniques that allow the audience to perceive the relationships 
between elements and sense what information goes together. 

relevant population those elements in the population most 
likely to have the information specified in the investigative 
questions. 

reliability a characteristic of measurement concerned with ac- 
curacy, precision, and consistency; a necessary but not suf- 
ficient condition for validity (if the measure is not reliable, it 
cannot be valid). 

reliability, equivalence a characteristic of measurement in 
which instruments can secure consistent results by the same 
investigator or by different samples. 

reliability, internal consistency a characteristic of an instru- 
ment in which the items are homogeneous. 

reliability, stability a characteristic of measurement in which 
an instrument can secure consistent results with repeated 
measurements of the same person or object. 

replication the process of repeating an experiment with differ- 
ent subject groups and conditions to determine the average 
effect of the IV across people, situations, and times. 

reporting study provides a summation of data, often recasting 
data to achieve a deeper understanding or to generate statis- 
tics for comparison. 

request for proposal (RFP) a formal bid request for research to 
be done by an outside supplier of research services. 


research briefing another term for the oral presentation; starts 
with a brief statement that sets the stage for the body of the 
findings and explains the nature of the project, how it came 
about, and what it attempted to do. This is followed by a dis- 
cussion of the findings that support it. Where appropriate, 
recommendations are stated in the third stage. 

research design the blueprint for fulfilling research objectives 
and answering questions. 

research process a sequence of clearly defined steps within a 
research study. 

research question(s) the hypothesis that best states the objec- 
tive of the research; the answer to this question would pro- 
vide the manager with the desired information necessary to 
make a decision with respect to the management dilemma. 

research report the document that describes the research proj- 
ect, its findings, analysis of the findings, interpretations, con- 
clusions, and, sometimes, recommendations. 

research variable see variable. 

residual the difference between the regression line value of Y and 
the real Y value; what remains after the regression line is fit. 

resistant statistics statistical measures relatively unaffected by 
outliers within a data set; e.g., median and quartiles. 

respondent a participant in a study; a.k.a. participant or subject. 

response error occurs when the participant fails to give a cor- 
rect or complete answer. 

return on investment (ROD) the calculation of the financial 
return for all organizational expenditures. 

right to privacy the participant’s right to refuse to be inter- 
viewed or to refuse to answer any questions in an interview. 

right to quality the sponsor’s right to an appropriate, value- 
laden research design and data handling and reporting 
techniques. 

right to safety the right of interviewers, surveyors, experi- 
menters, observers, and participants to be protected from any 
threat of physical or psychological harm. 

rotation in principal components analysis, a technique used to 
provide a more simple and interpretable picture of the rela- 
tionships between factors and variables. 

rule of thirds the method by which photographers compose 
their shots in the viewfinder using real or imaginary cross- 
hairs that divide field of view into thirds, vertically and hori- 
zontally to create a balanced visual. 

rule of three a presentation organizing device that uses trios, 
triplets, or triads in organizing support for an argument. 


sample a group of cases, participants, events, or records 
consisting of a portion of the target population, carefully 
selected to represent that population; see also pilot test, data 
mining. 

sample frame list of elements in the population from which the 
sample is actually drawn. 

sample statistics descriptors of the relevant variables computed 
from sample data. 

sampling the process of selecting some elements from a popu- 
lation to represent that population. 

sampling error error created by the sampling process; the error 
not accounted for by systematic variance. 

scaling the assignment of numbers or symbols to an indicant of 
a property or objects to impart some of the characteristics of 
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the numbers to the property; assigned according to value or 
magnitude. 

scalogram analysis a procedure for determining whether a set 
of items forms a unidimensional scale; used to determine if 
an item is appropriate for scaling. 

scatterplot a visual technique that depicts both the direction 
and the shape of a relationship between variables 

scientific method systematic, empirically based procedures for 
generating replicable research; includes direct observation of 
phenomena; clearly defined variables, methods, and proce- 
dures; empirically testable hypotheses; the ability to rule out 
rival hypotheses; and statistical rather than linguistic justifi- 
cation of conclusions. 

scope the breadth and depth of topic coverage of a secondary source 
(by time frame, geography, criteria for inclusion, etc.); one of the 
five factors for evaluating the quality of secondary sources. 

screen question question to qualify the participant’s knowledge 
about the target questions of interest or experience necessary 
to participate. 

script a written version of the introduction, arguments, con- 
clusion, and recommendations used in preparing for a 
presentation. 

search query the combination of keywords and connectors, op- 
erators, limiters, and truncation and phrase devices used to 
conduct electronic searches of secondary data sources; a.k.a. 
search statement. 

search statement see search query. 

secondary data results of studies done by others and for dif- 
ferent purposes than the one for which the data are being 
reviewed. 

secondary sources interpretations of primary data generally 
without new research. 

self-administered survey an instrument delivered to the par- 
ticipant via personal (intercept) or nonpersonal (computer- 
delivered, mail-delivered) means that is completed by the 
participant without additional contact with an interviewer. 

semantic differential (SD) scale measures the psychological 
meanings of an attitude object and produces interval data; 
uses bipolar nouns, noun phrases, adjectives, or nonverbal 
stimuli such as visual sketches. 

semantic mapping a projective technique in which participants 
are presented with a four-quadrant map in which different 
variables anchor the two different axes; they then spatially 
place brands, product components, or organizations within 
the four quadrants. 

semistructured interview an IDI that starts with a few spe- 
cific questions and then follows the individual’s tangents of 
thought with interviewer probes; questions generally use an 
open-ended response strategy. 

sensitive attitude one that a holder feels uncomfortable sharing 
with others. 

sensory sorts participants are presented with scents, textures, 
and sounds, usually verbalized on cards, and asked to arrange 
them by one or more criteria as they relate to a brand, prod- 
uct, event, etc. 

sentence completion a projective technique in which partici- 
pants are asked to complete a sentence related to a particular 
brand, product, event, user group, etc. 

sentence outline report planning format; uses complete sentences 
rather than key words or phrases to draft each report section. 
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sequential interviewing an IDI technique in which the partici- 
pant is asked questions formed around an anticipated series 
of activities that did or might have happened; used to stimu- 
late recall within participants of both experiences and emo- 
tions; a.k.a. chronologic interviewing. 

sequential sampling see double sampling. 

simple category scale a scale with two mutually exclusive re- 
sponse choices; a.k.a. dichotomous scale. 

simple observation unstructured and exploratory observation 
of participants or objects. 

simple prediction when we take the observed values of X to 
estimate or predict corresponding Y values; see also regres- 
sion analysis. 

simple random sample a probability sample in which each ele- 
ment has a known and equal chance of selection. 

simplicity a design principle that emphasizes reducing clutter 
and advocates using only the information and visual tech- 
niques necessary to convey the data, idea, or conclusion. 

simulated test market (STM) test of a product conducted in a 
laboratory setting designed to simulate a traditional shopping 
environment. 

simulation a study in which the conditions of a system or pro- 
cess are replicated. 

skewness a measure of a data distribution’s deviation from 
symmetry; if fully symmetrical, the mean, median, and mode 
are in the same location. 

skip interval interval between sample elements drawn from a 
sample frame in systematic sampling. 

skip pattern instructions designed to route or sequence the 
participant to another question based on the answer to a 
branched question. 

slope (B,) the change in Y for a 1-unit change in X; one of two 
regression coefficients. 

snowball sampling a nonprobability sampling procedure in 
which subsequent participants are referred by current sample 
elements; referrals may have characteristics, experiences, or 
attitudes similar to or different from those of the original sam- 
ple element; commonly used in qualitative methodologies. 

solicited proposal proposal developed in response to an RFP. 

Somers’s d a measure of association for ordinal data that com- 
pensates for “tied” ranks and adjusts for direction of the in- 
dependent variable. 

sorting participants sort cards (representing concepts or con- 
structs) into piles using criteria established by the researcher. 

sound reasoning the basis of sound research, based on finding 
correct premises, testing connections between facts and as- 
sumptions, and making claims based on adequate evidence. 

source evaluation the five-factor process for evaluating the 
quality and value of data from a secondary source; see also 
purpose, scope, authority, audience, and format. 

spatial behavior how humans physically relate to one another. 

spatial observation the recording of how humans physically 
relate to each other; see also proxemics. 

spatial relationships study an observation study that records how 
humans physically relate to each other (see also proxemics). 

speaker note cards a brief version of a presentation, in outline 
or keyword form; may be written on index cards; used for re- 
minding the presenter of the organization of the presentation. 

Spearman’s rho correlates ranks between two ordered vari- 
ables; an ordinal measure of association. 


specialty researcher establishes expertise in one or a few research 
methodologies; these specialties usually are based on methodol- 
ogy, process, industry, participant group, or geographic region; 
often assists other research firms to complete projects. 

specific instance a critical incident selected to prove an over- 
arching claim whereby specifics are translated into more gen- 
eral principles; not as detailed as stories; a form of inductive 
reasoning. 

specification error an overestimation of the importance of the 
variables included in a structural model. 

sponsor nondisclosure a type of confidentiality; when the 
sponsor of the research does not allow revealing of its 
sponsorship. 

spreadsheet a data-entry software application that arranges 
data cases or records as rows, with a separate column for each 
variable in the study. 

stability characteristic of a measurement scale if it provides 
consistent results with repeated measures of the same person 
with the same instrument. 

standard deviation (s) a measure of spread; the positive square 
root of the variance; abbreviated std. dev.; affected by ex- 
treme scores. 

standard error of the mean the standard deviation of the dis- 
tribution of sample means. 

standard normal distribution the statistical standard for de- 
scribing normally distributed sample data; used with inferen- 
tial statistics that assume normally distributed variables. 

standard score (Z score) conveys how many standard devia- 
tion units a case is above or below the mean; designed to 
improve compatibility among variables that come from dif- 
ferent scales yet require comparison; includes both linear ma- 
nipulations and nonlinear transformations. 

standard test market real-time test of a product through exist- 
ing distribution channels. 

standardized coefficients regression coefficients in standard- 
ized form (mean = 0) used to determine the comparative im- 
pact of variables that come from different scales; the X values 
restated in terms of their standard deviations (a measure of 
the amount that Y varies with each unit change of the associ- 
ated X variable). 

Stapel scale a numerical scale with up to 10 categories (7 posi- 
tive, 7 negative) in which the central position is an attribute. 
The higher the positive number, the more accurately the at- 
tribute describes the object or its indicant. 

statistical significance an index of how meaningful the 
results of a statistical comparison are; the magnitude of dif- 
ference between a sample value and its population value; 
the difference is statistically significant if it is unlikely 
to have occurred by chance (represent random sampling 
fluctuations). 

statistical study a study that attempts to capture a population’s 
characteristics by making inferences from a sample’s charac- 
teristics; involves hypothesis testing and is more comprehen- 
sive than a case study. 

statistics (in presentations) numerical data used in the collec- 
tion, analysis, and interpretation of data, but also found in of 
the data collection planning, measurement, and design; ex- 
pected in research presentations. 

stem-and-leaf display a tree-type frequency distribution for 
each data value, without equal interval grouping. 


stepwise selection in modeling and regression, a method for 
sequentially adding or removing variables from a regression 
model to optimize R’; combines forward selection and back- 
ward elimination methods. 

stories a type of supporting material used in a presentation that 
tells the particulars of an act or occurrence or course of events; 
is most powerful when involving personal experience. 

strategy the general approach an organization will follow to 
achieve its goals. 

stratified random sampling probability sampling that includes 
elements from each of the mutually exclusive strata within a 
population. 

stratified sampling, disproportionate a probability sampling 
technique in which each stratum’s size is not proportionate to 
the stratum’s share of the population; allocation is usually based 
on variability of measures expected from the stratum, cost of 
sampling from a given stratum, and size of the various strata. 

stratified sampling, proportionate a probability sampling 
technique in which each stratum’s size is proportionate to the 
stratum’s share of the population; higher statistical efficiency 
than a simple random sample. 

stress index an index used in multidimensional scaling that 
ranges from 1| (worst fit) to 0 (perfect fit). 

structural equation modeling (SEM) uses analysis of covari- 
ance structures to explain causality among constructs. 

structured interview an IDI that often uses a detailed interview 
guide similar to a questionnaire to guide the question order; 
questions generally use an open-ended response strategy. 

structured response participant’s response is limited to spe- 
cific alternatives provided; a.k.a. closed response. 

summated rating scale category of scales in which the partici- 
pant agrees or disagrees with evaluative statements; the Lik- 
ert scale is most known of this type of scale. 

supergroup a group interview involving up to 20 people. 

survey a measurement process using a highly structured inter- 
view; employs a measurement tool called a questionnaire, 
measurement instrument, or interview schedule. 

survey via personal interview a two-way communication initi- 
ated by an interviewer to obtain information from a partici- 
pant; face-to-face, phone, or Internet. 

symmetrical relationship occurs when two variables vary to- 
gether but without causation. 

syndicated data provider tracks the change of one or more 
measures over time, usually in a given industry. 

synergy the process at the foundation of group interviewing that 
encourages members to react to and build on the contribu- 
tions of others in the group. 

systematic error error that results from a bias; see also system- 
atic variance. 

systematic observation data collection through observation 
that employs standardized procedures, trained observers, 
schedules for recording, and other devices for the observer 
that mirror the scientific procedures of other primary data 
methods. 

systematic sampling a probability sample drawn by applying 
a calculated skip interval to a sample frame; population (NV) 
is divided by the desired sample (n) to obtain a skip interval 
(k). Using a random start between 1| and k, each kth element 
is chosen from the sample frame; usually treated as a simple 
random sample but statistically more efficient. 
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systematic variance the variation that causes measurements to 
skew in one direction or another. 


t distribution a normal distribution with more tail area than that 
in a Z normal distribution. 

t-test a parametric test to determine the statistical significance 
between a sample distribution mean and a population pa- 
rameter; used when the population standard deviation is un- 
known and the sample standard deviation is used as a proxy. 

tactics specific, timed activities that execute a strategy. 

target population those people, events, or records that contain 
the desired information for the study that determine whether 
a sample or a census should be selected. 

target question measurement question that addresses the core 
investigative questions of a specific study; these can be struc- 
tured or unstructured questions. 

target question, structured a measurement question that pres- 
ents the participant with a fixed set of categories per variable. 

target question, unstructured measurement question that pre- 
sents the participant with the context for participant-framed 
answers; a.k.a. open-ended question, free-response question 
(nominal, ordinal, or ratio data). 

tau (t) a measure of association that uses table marginals to re- 
duce prediction errors, with measures from 0 to 1.0 reflecting 
percentage of error estimates for prediction of one variable 
based on another variable. 

tau b (7, ) arefinement of gamma for ordinal data that considers 
“tied’ pairs, not only discordant and concordant pairs (values 
from —1.0 to +1.0); used best on square tables (one of the 
most widely used measures for ordinal data). 

tau c(t, ) arefinement of gamma for ordinal data that considers 
“tied’ pairs, not only discordant and concordant pairs (val- 
ues from —1.0 to +1.0); useful for any-size table (one of the 
most widely used measures for ordinal data). 

technical report a report written for an audience of 
researchers. 

telephone focus group a type of focus group in which par- 
ticipants are connected to the moderator and each other by 
modern teleconferencing equipment; participants are often 
in separate teleconferencing facilities; may be remote- 
moderated or -monitored. 

telephone interview a study conducted wholly by telephone 
contact between participant and interviewer. 

telephone survey a structured interview conducted via tele- 
phone. 

10-minute rule varying a presentation’s content on 10-minute 
intervals with videos, demonstrations, questions, and other 
means to allow the brain to avoid boredom/fatigue and seek 
new stimuli. 

tertiary sources aids to discover primary or secondary sources, 
such as indexes, bibliographies, and Internet search engines; 
also may be an interpretation of a secondary source. 

testimony (expert opinion) opinions of recognized experts 
who possess credibility for your audience on a topic; used as 
support or proof. 

test market a controlled experiment conducted in a carefully 
chosen marketplace (e.g., website, store, town, or other geo- 
graphic location) to measure and predict sales or profitability 
of a product. 
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test unit an alternative term for a subject within an experiment 
(a person, an animal, a machine, a geographic entity, an ob- 
ject, etc.). 

Thematic Apperception Test a projective technique in which 
participants are confronted with a picture (usually a photo- 
graph or drawing) and asked to describe how the person in 
the picture feels and thinks. 

theoretical sampling a nonprobability sampling process in 
which conceptual or theoretical categories of participants de- 
velop during the interviewing process; additional participants 
are sought who will challenge emerging patterns. 

theory a set of systematically interrelated concepts, definitions, 
and propositions that are advanced to explain or predict phe- 
nomena (facts); the generalizations we make about variables 
and the relationships among variables. 

3-D graphic a presentation technique that permits a graphical 
comparison of three or more variables; types: column, rib- 
bon, wireframe, and surface line. 

three-point speech variations on the rule of three in speech or- 
ganization that may include introduction—body—conclusion; 
introduction—three best supporting points—conclusion; three 
stories; or, other devices in threes. 

time sampling the process of selecting certain time points or 
time intervals to observe and record elements, acts, or condi- 
tions from a population of observable behaviors or conditions 
to represent the population as a whole; three types include 
time-point samples, time-interval samples, and continuous 
real-time samples. 

topic outline report planning format; uses key words or phrases 
rather than complete sentences to draft each report section. 

treatment the experimental factor to which participants are 
exposed. 

treatment levels the arbitrary or natural groupings within the 
independent variable of an experiment. 

triad a group interview involving three people. 

trials repeated measures taken from the same subject or participant. 

triangulation research design that combines several qualita- 
tive methods or qualitative with quantitative methods; most 
common are simultaneous QUAL/QUANT in single or mul- 
tiple waves, sequential QUAL-QUANT or QUANT-QUAL, 
sequential QUAL-QUANT-QUAL. 

truncation a search protocol that allows a symbol (usually “?” 
or “*’’) to replace one or more characters or letters in a word 
or at the end of a word root. 

two-independent-samples tests parametric and nonparametric 
tests used when the measurements are taken from two sam- 
ples that are unrelated (Z test, t-test, chi-square, etc.). 

two-related-samples tests parametric and nonparametric tests 
used when the measurements are taken from closely matched 
samples or the phenomena are measured twice from the same 
sample (t-test, McNemar test, etc). 

two-stage design a design in which exploration as a distinct 
stage precedes a descriptive or causal design. 

two-tailed test a nondirectional test to reject the hypothesis 
that the sample statistic is either greater than or less than the 
population parameter. 

Type I error error that occurs when one rejects a true null hy- 
pothesis (there is no difference); the alpha (a) value, called 
the level of significance, is the probability of rejecting the 
true null hypothesis. 


Type II error error that occurs when one fails to reject a false 
null hypothesis; the beta (8) value is the probability of failing 
to reject the false null hypothesis; the power of the test 1 — B 
and is the probability that we will correctly reject the false 
null hypothesis. 


unbalanced rating scale has an unequal number of favorable 
and unfavorable response choices. 

unforced-choice rating scale provides participants with an op- 
portunity to express no opinion when they are unable to make 
a choice among the alternatives offered. 

unidimensional scale instrument scale that seeks to measure 
only one attribute of the participant or object. 

unobtrusive measures a set of observational approaches that 
encourage creative and imaginative forms of indirect ob- 
servation, archival searches, and variations on simple and 
contrived observation, including physical traces observation 
(erosion and accretion). 

unsolicited proposal a suggestion by a contract researcher for 
research that might be done. 

unstructured interview a customized IDI with no specific 
questions or order of topics to be discussed; usually starts 
with a participant narrative. 

unstructured response participant’s response is limited only 
by space, layout, instructions, or time; usually free-response 
or fill-in response strategies. 

utility score a score in conjoint analysis used to represent each 
aspect of a product or service in a participant’s overall prefer- 
ence ratings. 


validity a characteristic of measurement concerned with the ex- 
tent that a test measures what the researcher actually wishes 
to measure; and that differences found with a measurement 
tool reflect true differences among participants drawn from 
a population. 

validity, construct the degree to which a research instrument is 
able to provide evidence based on theory. 

validity, content the extent to which measurement scales pro- 
vide adequate coverage of the investigative questions. 

validity, criterion-related the success of ameasurement scale for 
prediction or estimation; types are predictive and concurrent. 

variability term for measures of spread or dispersion within a 
data set. 

variable (research variable) a characteristic, trait, or attribute 
that is measured; a symbol to which values are assigned; in- 
cludes several different types: continuous, control, decision, 
dependent, dichotomous, discrete, dummy, extraneous, inde- 
pendent, intervening, and moderating variables. 

variance a measure of score dispersion about the mean; calcu- 
lated as the squared deviation scores from the data distribu- 
tion’s mean; the greater the dispersion of scores, the greater 
the variance in the data set. 

videoconferencing focus group a type of focus group in which 
researchers use the videoconference facilities of a firm to con- 
nect participants with moderators and observers; unlike tele- 
phone focus groups, participants can see each other; can be 
remotely moderated, and in some facilities can be simultane- 
ously monitored by client observers via Internet technology. 


virtual test market a test of a product using a computer simula- 
tion of an interactive shopping experience. 

visibility the design principle that visual support materials must 
be sized and placed in the presentation setting to facilitate the 
audience’s ability to see and read the content. 

visitor from another planet a projective technique (imagi- 
nation exercise) in which participants are asked to assume 
that they are aliens and are confronting the product for the 
first time; they then describe their reactions, questions, and 
attitudes about purchase or retrial. 

visual aids presentation tools used to facilitate understanding of 
content (e.g., chalkboards, whiteboards, handouts, flip charts, 
overhead transparencies, slides, computer-drawn visuals, 
computer animation). 

visual learners people who learn through seeing; about 40 per- 
cent of the audience; implies the need to include visual imag- 
ery, including graphs, photographs, models, etc., in research 
presentations. 

visual preparation the design principle that a presenter should 
conceptualize the visual support materials on paper before 
composing the digital versions. 

visualization the process of developing and organizing support 
materials that help the audience share in your understanding 
of the data. 

voice recognition computer systems programmed to record ver- 
bal answers to questions. 


Web-based questionnaire a measurement instrument both 
delivered and collected via the Internet; data processing is 
ongoing. Two options currently exist: proprietary solutions 
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offered through research firms and off-the-shelf software 
for researchers who possess the necessary knowledge and 
skills; a.k.a. online survey, online questionnaire, Internet 
survey. 

Web-delivered presentation one that involves the use of a 
Web presentation platform, a presenter who remotely con- 
trols the delivery of the presentation, and an invited audience 
who participates via the Web from their office or a Web- 
equipped room. 

Web-enabled test market test of a product using online 
distribution. 

whitespace a design principle of leaving empty, uncluttered 
space surrounding important key visuals and text; permits 
audience to achieve a visual focus. 

word or picture association a projective technique in which 
participants are asked to match images, experiences, emo- 
tions, products and services, and even people and places to 
whatever is being studied. 


Z distribution the normal distribution of measurements 
assumed for comparison. 

Z score see standard score. 

Z test a parametric test to determine the statistical significance 
between a sample distribution mean and a population param- 
eter; employs the Z distribution. 
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Ltd, 223 

Research Triangle Institute (RTI 
Interational), 232 

reviewcentre.com, 394 

RFG (Research for Good), 344 

Rider University, 100 

Robert Wood Johnson Foundation, 606 

Robinson Organization, 484 

Rocket Science Group LLC, 198 

Romande Conti St. Vivant, 480 

RTG (Research Through Gaming) 
Ltd., 223 

RTI International (Research Triangle 
Institute), 232 

Rypple, 145 


Ss 


Saleforce.com, 103 

Samsung, 398 

SAP (Systems Applications and 
Products), 397, 422 

SAS, 253, 423, 443 

SaySo for Good, 344 

SCHIP (State Children’s Health 
Insurance Program), 578 

Seaport Surveys, 37 

Searchwise, 100 

Serta, 340 

Sharp, 398 

SHL, 255 

ShopAlert, 30 

Simplement, Inc., 397 

Singapore Airlines, 289 

SIS International Research, 134 

SkyDrive (Microsoft cloud), 103 

Skype services, 164 

SnapFish, 183 

Society for Human Resource 
Management, 41 
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Society of Competititve Intelligence 
Professionals, 42 

SONY, 188, 398 

Southwest Airlines, 12 

Spherion Inc., 319 

SRI International, 384 

Stanford University, 187, 554 

Starbucks, 547, 582 

Stars Insight, 5 

State Children’s Health Insurance 
Program (SCHIP), 578 

State Farm Dangerous Intersection 
Study, 127 

State Farm Insurance, 582 


Statistical Abstract of the United States, 99 


Sun Microsystems, 123 

Super Bowl, 254 

Super Searchers Web Page, 100 
Survey Monkey, 103, 134, 273 


Survey Research Center, University of 


Michigan, 233 
Survey Sampling Inc., 229 
Swatch’s Beat Time, 253 
Swift & Co., 60 
Sybase, 170 


T 


Target Stores, Inc., 84 

TCS Management Group, 577 

Team One Advertising, 544, 579 

Technorati, 97 

TechTarget, 2 

The Beard Institute, Duquesne 
University, 45 

The Container Store, 79 

The Taylor Group, 582 

Tigris Consulting, 378 

Time magazine, 515, 564 

Time Warner Inc., 255, 418 

TiVo, 254 


TNS Infratest (TNS), 5 

Toyota Motor Co., 430, 433 
Travel Industry Magazine, 454 
Troy-Bilt, 434, 580 

TrustE, 30, 34 

Twitter, 145 
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U 


K Department for Work and 
Pensions, 223 
HC Inc., 410 


Jnited Airlines, 212 


nited Latin American Citizens, 131 

nited States Tennis Association 
(USTA), 582 

niversity of Arkansas, 194 

niversity of British Columbia 
(Canada), 65 

niversity of California, San Diego, 65 

niversity of Michigan, 233 

niversity of Wisconsin-Madision, 538 

PS, 12 


JS Army, 465 
JS Bureau of Labor Statistics, 99 
JS Census Bureau, 194, 331, 339, 


343, 418 

S Department of Agriculture 
(USDA), 60 

S Department of Commerce, 226 

S Department of Defense, 494 

S Department of Health and Human 
Services, 42 


JS government, 98, 103 
JS Open, 581 


S Transportation Security 
Administration, 309 
SA Today, 535 


uSamp, 336 


U 


STA (United States Tennis 
Association), 582 


V 


Vanity Fair, 120 

VC Experts, 99 

Verint, 298, 502 

VHA Inc., 410 

Vicale Corporation, 579 

Victoria Transportation Policy 
Institute, 99 

Vienna University of Economica 
and Business Administration 
(Austria), 177 

Visa, 582 

Volkswagen, 583 


WwW 


Walmart Stores, Inc., 84, 182, 189 
Walsworth Publishing, 264 
Warner Brothers, 537 

WebEx, 164, 547 

WebMD, 112 
WeddingChannel.com, 194 
Wendy’s International, 204 
Wired, 447 

Wirthlin Worldwide, 578, 606 
Wittenberg University, 580 


Y 


Yahoo!, 100, 103, 352, 583 

Yale School of Management, 394 
Yankelovich Partners, 240 
YouTube, 183, 554 

Yum! Brands, 12, 374 


Zz 


Zanthus, 404 

ZDnet.com, 394 

Zoomerang, 134 

ZQ Intelligence (Luth Research), 135 


A 


Abstractions, data as, 85 
A/B tests, for Web sites, 447 
Accretion, as unobtrusive measure, 
184, 187 
Accuracy 
recall, 546-547 
sampling, 340-341 
Acquiescence, bias from, 224 
Actionable research development, 108-118 
investigative questions, 113-117 
management question, 108-112 
measurement questions, 118 
research questions, 112-113 
Action research, 166 
Active agent technology, 33 
Activity analysis, 174 
Administrative questions, 302 
Advertising, 254, 484 
Affectively based attitudes, 269-270 
After-only research designs, 204 
AID (Automatic interaction 
detection), 423 
Alternative hypothesis, 432; see also 
Hypothesis testing 
Ambiguities techniques, 155 
Analogy, in presentations, 553 
Anonymity, of mail surveys, 229 
ANOVA (analysis of variance) 
Friedman two-way, 462 
F test of regression model and, 488-489 
one-way, 453-457 
repeated measures, 460-462 
two-way, 458-460 
Applied research, 15 
Appropriate knowledge, principle of, 555 
Appropriateness, in coding, 383 
Apps-based research, 83 
A priori contrasts, 457 
Area charts, 534 
Area sampling, 356 
Areas of the standard normal distribution 
(table), 620 
Argument, 66 
Aristotle’s three persuasive communication 
principles, 541-543 
Artifact correlation, 476-477 
ASR (automatic speech recognition), 233 
Assessment, 107, 255 
Association, measures of, 466-501 
bivariate correlation analysis, 469-479 
interpretation of, 476-479 
Pearson’s product moment 
coefficient r, 469-470, 473-476 
scatterplots, 470-473 


bringing research to life vignettes, 
467-468 
nonparametric, 490-498 
nominal data measures, 490-494 
ordinal data measures, 494-498 
simple linear regression, 479-490 
application of, 480-482 
goodness of fit testing, 487-490 
least squares method, 482-485 
model of, 479-480 
predictions from, 485-487 
Asymmetrical relationships of 
variables, 138 
Attention span, 419 
Attitudes; see Measurement scales 
Audience analysis, 544-545 
Audience Memory Curve, 546 
Auditory learning styles, 546 
Authority figure techniques, 155 
Authorization letter, of reports, 508 
Automated telephone surveys, 309 
Automatic interaction detection (AID), 423 
Automatic speech recognition (ASR), 233 
Available criteria, 258-259 


Balanced rating scales, 272 
Bar charts, 407-408, 535 
Bar codes, for data entry, 394-395 
Basic research, 15 
Bayesian statistics, 430 
Behavior, attitude relationship to, 270 
Behavioral observations, 175, 185-186 
Behavior standards, ethics as, 28 
Beliefs, attitudes and, 269 
Belmont Report of 1979, 40 
Benchmarking, 150 
Benefit chain techniques, 155 
Benefits of research, 29-30 
Bias 
data interpretation and, 39 
experimental settings and, 203 
freedom from, 258 
halo effect, 274-275 
physical presence, 220-221 
response, 224 
social desirability, 224 
split-ballot technique to prevent, 310 
wording with, 332 
Bibliographies, 98, 514 
Biel Meridian timeline, 253 
Big business, criticism of, 6 
Bing search engine, 422 
Biometric response, 83 
Biometrics, 35 


Bivariate correlation analysis, 469-479 
interpretation of, 476-479 
Pearson’s product moment coefficient r, 
469-470, 473-476 
scatterplots, 470-473 
Bivariate normal distribution of Pearson’s 
product moment coefficient r, 473 
Bivariate relationships, 135 
Black-letter law, 30 
Blatherers, in focus groups, 160 
Blind experiments, 197 
Blogs, 97, 105, 223 
Body orientation, in presentations, 
565-566 
Box-and-whisker plot, 415 
Boxplots, 415-418 
Brain scans, 177 
Branching questions, 317 
Brand mapping techniques, 156 
Brand power, 350-351 
Bringing Research to Life vignettes 
association, measures of, 467-468 
client background, 49-50 
data display and examination, 405 
data preparation and description, 375 
ethics, 27 
experiments, 191 
hypothesis testing, 429 
measurement, 245-246 
measurement scales, 267 
observations, 171 
oral presentations, 539 
proposal reviews, 34 
qualitative research, 143 
questionnaires, 295 
reports, 503 
research design, 123-124 
research question, 75, 93 
sampling, 337-338 
sponsor background, 49-50 
surveys, 215-216 
Budgets, research, 80-82; see also Cost; 
Proposal, research 
Business intelligence (BD), 9, 99, 237 
Business Outcomes Study Report of 2012 
(SHL), 255 
Business research 
competitive advantage, 7-10 
decision makers, information-based, 
11-13 
“good research” characteristics, 15-18 
proposal reviews, 3-4 
reasons to study, 4-7 
research process, 13-15 
study examples, 18—22 
unneeded or inappropriate, 38 
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Cc 


CADAC (computer-assisted data 
collection), 224 
Call-filtering systems, telephone surveys 
and, 233-234, 237 
Capacity limitations, principle of, 555 
CAPI (computer-assisted personal 
interviewing), 156, 238, 578 
Cartoon dialog techniques, 155 
Case studies, 128, 165 
CASIC (computer-assisted survey 
information collection), 224 
CASIs (computer-assisted self- 
interviews), 226 
Categories, of measurement 
questions, 302 
Categorization, 271, 379 
CATI (computer-assisted telephone 
interviewing), 233, 239, 578 
Causal-explanatory studies, 127 
Causal hypotheses, 59 
Causal methods, 192 
Causal-predictive studies, 127 
Causal study design, 136-139 
Cell phones, 232, 237; see also Mobile 
devices; Smart phones 
Census, 84, 338-339 
Central editing, of data, 377-379 
Central limit theorem, 366 
Central tendency, measures of, 252, 274, 
400-401 
Checklists 
measurement scale as, 274, 277 
observation, 179-180 
questions on, 307, 311 
Children, as research subjects, 31; 
see also Participants, research 
Children’s Online Privacy Protection Act 
(COPPA), 32 
Chi-square (x7) test 
nonparametric k-independent samples 
situation, 460 
nonparametric measures of association, 
490-49 1 
nonparametric one-sample situation, 
445-446 
nonparametric two-independent- 
samples situation, 448-450 
Clarity, for presentations, 561 
Classical statistics, 430 
Classification, 107, 302 
Clear and to the Point (Kosslyn), 555 
Click-through activity, 211 
Closed circuit television, 35 
Closed questions, coding, 380-381 
Closed response, 306 
CloseUp feature 
dirty data problem, 378 
instrument design, 321-322 


measurement scales, 288-289 
observation, 185-186 
quasi-experiments, 208 
report example, 518-528 
research proposal, 114-117 
search engines, 350-351 
tables, 412-414 
wildcat survey, 134 
Cloud computing, 103 
Clustering, 106 
Cluster sampling, 355-357 
Clutter, in presentations, 564 
Cochran Q test, 462 
Code of ethics, 40; see also Ethics 
Coding data, 379-391 
closed questions, 380-381 
codebook construction, 380 
content analysis, 384-387, 442 
“don’t know” responses, 387-389 
missing data, 389-391 
open-ended questions, 382 
overview, 379-380 
rules of, 383-384 
Coefficient of determination, 474, 489 
Coefficients of variation, 254 
Cognitively based attitudes, 269-270 
Cohort group longitudinal studies, 128 
Common variance, 474-475 
Communication approach, 216-226 
choosing, 224-226 
description of, 216-219 
errors in, 219-224 
management question hierarchy and, 
298-299 
persuasive, 541-543 
Communication studies, 127 
Communities of practice, 112 
Co-moderators, in focus groups, 160 
Comparative measurement scale, 285 
COMPASS methodology (Harris 
Interactive), 287 
Compatibility, for presentations, 559 
Competitive advantage, 7-10 
Complex probability sampling, 352-358 
Component sort techniques, 155, 161 
Comprehensibility, of reports, 515-516 
Computer-assisted data collection 
(CADAC), 224 
Computer-assisted personal interviewing 
(CAPD), 156, 238, 578 
Computer-assisted self-interviews 
(CASIs), 226 
Computer-assisted survey information 
collection (CASIC), 224 
Computer-assisted telephone interviewing 
(CATD, 233, 239, 578 
Computer-assisted telephone survey, 233 
Computer chips, implantable, 60 
Computer-delivered surveys, 226 
Conatively based attitudes, 270 


Concealment, in observation, 177-178 
Concentration, of observers, 181 
Concepts, research, 50-52, 248-249 
Conceptual schemes, 52 
Conclusions, from research, 18 
Concordant observations, 495 
Concurrent criterion-related validity, 257 
Conducting experiments, 194-200 
controlling environment, 197 
data analysis, 199-200 
design choice, 197-198 
participant selection, 199 
pilot testing, 199 
treatment levels, 195-197 
variable selection, 195 
Conducting observations 
content specification, 180 
data collection, 181-184 
observer training, 181 
study type, 178-180 
Confidence intervals, 365—366, 368 
Confidentiality 
participant, 34, 39 
sponsor, 36-37 
Confirmatory data analysis (EDA), 406 
Confounding variables (CFVs), 57 
Conjoint analysis, spreadsheet displays 
for, 421 
Connectivity increases, 6 
Connotative meanings, measuring, 289 
Consistency, internal, 261 
Constant-sum measurement scales, 
276, 284 
Constructs 
attitude as, 270 
description of, 52-53 
object of measurement as, 248-249 
Construct validity, 257, 259 
Consultancy skills for research, 10 
Consumer packaged goods (CPG) 
companies, 182 
Content, of measurement questions, 
302-305, 328-333 
Content analysis 
in coding data, 384-387 
of focus group comments, 165-166 
hypothesis testing and, 442 
of observations, 174 
of Web-posted feedback, 394 
Content specification, of observations, 
180 
Content validity, 257-258 
Continuous variables, 55 
Contrast, for presentations, 558-559 
Control 
prediction and, 22 
in research designs, 137 
sampling, 360 
Control groups, in experiments, 197 
Control variables (CVs), 57 


Convenience, measurement, 262 
Convenience nonprobability 
sampling, 359 
Convenience of experiments, 193 
Convenience sampling, 152 
Convergent interviewing, 158 
Convergent validity, 259 
Conversational information, 13 
Corporate culture, 28, 38 
Corporate restructuring, 10 
Correlational hypotheses, 59 
Correlational studies, 22, 134, 252; 
see also Association, measures of 
Correspondence analysis, 106 
Cost 
experiments, 193 
personal interview, 238 
sample size, 349 
sample versus census, 338-339 
surveys, 228 
Covariance, 137, 474 
CPG (consumer packaged goods) 
companies, 182 
CPM (critical path method), 125, 
599-600 
Criterion-related validity, 257-259 
Criterion variables, 55 
Critical incident technique, 158 
Critical path method (CPM), 125, 
599-600 
Critical values 
of the chi-square distribution 
(table), 622 
of D in the Kolmogorov-Smirnov 
one-sample test (table), 623 
of D in the Kolmogorov-Smirnov 
two-samples test for large samples 
(two-tailed, table), 625 
of F distribution for o = .05 (table), 
626-627 
of T in the Wilcoxon matched-pairs test 
(table), 623 
of U in Mann-Whitney test (table), 628 
Cross-sectional studies, 128 
Cross-tabulation, 419-425 
overview, 135 
percentages in, 420-423 
table-based analysis, 423-425 
Cross-time-zone chats, 253 
Crowdsourcing, 144 
CUE methodology (Primary Insights, 
Inc.), 156 
Culture 
corporate, 28, 38 
interviews and, 158 
measurement scale metrics and, 274 
Cumulative measurement scales, 
289-290 
Custom-designed measurement 
questions, 118 


Customer choice analysis, spreadsheet 
displays for, 421 

Customer lifetime value analysis, 
spreadsheet displays for, 421 

Customer satisfaction measurement, 
87, 280 

Cyberspace, data collection in, 35; 
see also Internet 

Cynics, in focus groups, 160 


Dashboards, online, 264 
Data 
analysis and interpretation of, 86 
CADAC (computer-assisted data 
collection), 224 
CASIC (computer-assisted survey 
information collection), 224 
changing or falsifying, 39 
collection of, 85-86, 127 
cost of collection, 7 
European Commission’s Data 
Protection Directive, 36 
experimental, 199-200 
hypothetical constructs from, 52 
integration of, 7 
Internet collection of, 35-36, 226 
measurement scales and, 272 
from observations, 181-184 
speed of collection, 339 
voice-of-consumer (VoC), 242 
Database development, 392 
Data display and examination, 404-427 
bringing research to life vignette, 405 
cross-tabulation, 419-425 
percentages in, 420-423 
table-based analysis, 423-425 
exploratory data analysis, 406-419 
boxplots, 415-418 
frequency tables, bar charts, and pie 
charts, 407-408 
histograms, 408-410 
mapping, 418-419 
overview, 406-407 
Pareto diagrams, 415 
stem-and-leaf displays, 411-414 
Data entry error, 220 
Data fields, 392 
Data files, 392 
Data marts, 102 
Data mining 
ethics for, 35-36 
evolution of, 103-105 
process of, 105-107 
Data preparation and description, 374403 
bringing research to life vignette, 375 
coding, 379-391 
closed questions, 380-38 1 
codebook construction, 380 
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content analysis in, 384-387 
“don’t know” responses, 387-389 
missing data, 389-391 
open-ended questions, 382 
overview, 379-380 
rules of, 383-384 
editing, 377-379 
entry 
digital, 394-395 
innovations for, 395 
keyboarding, 391-393 
optical recognition, 393-394 
voice recognition, 394 
statistical, 398-402 
central tendency measures, 
400-401 
distributions, 398-400 
shape measures, 402 
variability measures, 401-402 
Data records, 392 
Data reduction programs, 106 
Data visualization, 106 
Data warehouse, 102 
Dating services, online, 194 
DBM (disk-by-mail) survey, 226 
Debriefing participants, 32 
Deception, 30-31, 99 
Decision makers, information-based 
hierarchy of, 11-13 
research needs of, 17 
videoconferenced focus groups to 
convince, 163 
Decision rule, 82 
Decision support systems (DSS), 8-9 
Decision theory, 81-82 
Decision trees, 107 
Decision variable, 82 
Deduction, 66—70, 542 
Defined purpose, of research, 15-16 
Definitions, research, 52-55 
Delayed equivalent forms approach, for 
reliability, 261 
Demonstrations, in presentations, 554 
Departmental budgeting, 80 
Dependent variables (DV), 55-56, 
58, 192 
Descriptive hypotheses, 58-59 
Descriptive models, 63 
Descriptive statistics, 400 
Descriptive studies, 21, 127, 134-136 
Designs; see Research design 
Detail orientation, of observers, 181 
Dichotomous measurement scale, 
274-275 
Dichotomous questions, 308 
Dichotomous variables, 55 
Dictionaries, 98 
Digital camera monitoring, 35 
Digital data entry, 394-395 
Digital “natives” and “immigrants,” 418 
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Directional test, in hypothesis testing, 432 


Direct mail ads, 447 
Direct observation, 177 
Directories, 100 
Direct-to-device software, 33 
Dirty data problem, 378 
Disabilities, people with, 287 
Discordant observations, 495 
Discrete variables, 55 
Discriminability, principle of, 557 
Discriminant validity, 259 
Discussion guides, 153, 610-611 
Disengage type participants, 300 
Disk-by-mail (DBM) survey, 226 
Dispersion, 253, 369-370 
Disposition-behavior asymmetrical 
relationships, 138 
Disproportionate stratified sampling, 
354-355 
Distribution 
five-number summary in, 415 
normal, 398, 473, 620 
statistics for, 398-400 
Distribution-free statistics, 252 
DNA matching systems, 60 
Dominators, Cynics, and Wallflowers 
(Kahle), 160 
Dominators, in focus groups, 160 
Do-Not-Call Improvement Act of 
2007, 237 
Do Not Call registry, 235 
“Don’t know” responses, 387-389 
Double-barreled questions, 309, 328 
Double blind experiments, 197 
Double sampling, 357-358 
Drift, of observers, 181 
Dummy tables, for preliminary 
analysis, 300 
“Dynasty Drivers” (Yum! Brands), 12 


E-business rating services, 226 

E-commerce, 34 

Economy in measurement, 262 

EDA (exploratory data analysis); see 
Exploratory data analysis (EDA) 

Editing data, 377-379 

EEG (electroencephalography), 177 

80/20 rule, 415 

Electroencephalography (EEG), 177 

Electronic monitoring, 35 

Elusive quality of data, 85 

E-mail open rates, 198 

Emerging research techniques, 83, 
97, 164 

Empiricism, 66, 246, 249 

Employee turnover, 255 

Encyclopedias, 98-99 

Enthymeme, in appeals, 542 


Environment, control of, 197 
Equivalence, reliability as, 261 
Erosion, as unobtrusive measure, 
184, 187 
Error 
of central tendency, 274 
in communication approach, 219-224 
data entry, 220 
instrument, 256 
interviewer, 220-221 
leniency, 272, 274 
measurer, 256 
noncompletion, 229 
noncoverage, 346 
nonresponse, 346 
proportional reduction in error (PRE) 
statistics, 492-493, 495 
rater, 274-275 
response, 222—224, 256 
sampling, 220, 341 
situational factors in, 256 
social-desirability, 229 
in surveys, 221 
Type I error, in hypothesis testing, 
435-436 
Type II error, in hypothesis testing, 
435-438 
e-speed, 506 
Estimation models, 107 
Ethics, 26-47 
bringing research to life vignette, 27 
overview, 16, 28 
in participant treatment, 28-36 
benefits of research discussed, 29-30 
debriefing, 32 
deception issues, 30-31 
informed consent of, 31-32 
Internet data collection, 35-36 
privacy rights of, 32-34 
physicians and patients as research 
subjects, 184 
professional standards, 40-42 
researcher and team members, 39-40 
research sponsor and, 36-39 
resources on, 42-45 
sentiment analysis and, 105 
Ethnography, 144, 158, 179 
Ethos, in persuasive communication, 542 
European Commission’s Data Protection 
Directive, 36 
Evaluation 
of experiments, 193-194 
of information sources, 100-102 
methods of, 81-82 
by observations, 69 
of observations, 176 
of personal interview survey, 238 
of research proposals, 601-605 
of self-administered surveys, 226 
of telephone interviews, 232—233 


Event sampling, in observations, 181 
Examples, in presentations, 552 
Excel software, modeling by, 63, 421 
Executive summary, of reports, 508 
Exhaustiveness, in coding, 383 
Experience level, of observers, 181 
Experience survey, 131-132 
Experiments, 127, 190-213 
bringing research to life vignette, 191 
conducting, 194-200 
controlling environment, 197 
data analysis, 199-200 
design choice, 197-198 
participant selection, 199 
pilot testing, 199 
treatment levels, 195-197 
variable selection, 195 
evaluation of, 193-194 
overview, 192 
preexperimental design, 204—206 
quasi- or semi-experimental designs, 
207-210 
true experimental designs, 206-207 
validity, 201-204 
Expert groups, 157 
Expert interviews, 94 
Expert opinion, in presentations, 553 
Explanatory hypotheses, 59 
Explanatory studies, 22 
Exploration, 94-102 
evaluating information sources, 
100-102 
information levels, 96-97 
information sources, 97—100 
Exploratory data analysis (EDA), 106, 
406-419 
boxplots, 415-418 
frequency tables, bar charts, and pie 
charts, 407-408 
histograms, 408-410 
mapping, 418-419 
overview, 406-407 
Pareto diagrams, 415 
stem-and-leaf displays, 411-414 
Exploratory research phase, 94 
Exploratory studies, 126, 129-134 
Exposition, 66 
Ex post facto design, 127 
Ex post facto evaluation, 81 
Extemporaneous presentation, 562 
Extension strategies, in product life 
cycle, 62 
External research proposals, 595 
External validity, 201, 203-204 
Extralinguistic behavior, 175 
Extraneous variables (EVs), 
57, 193 
Eyeblink rates, 175 
Eye contact, 565-566 
Eye-tracking studies, 183 


F 


Facial recognition, 35 
Facial analysis, 83 
Factor analysis, 106 
Facts, in presentations, 552 
Factual level, for observation, 180 
Falsifying data, 220 
Favored-technique syndrome, 87-88 
Feedback, 79, 394 
Field conditions, for studies, 128 
Field editing, of data, 377 
Field experiments, 193, 207-210 
Financial statement analysis, 174 
Findings, nondisclosure of, 37; see also 
Reports, research 
Finger printing, 35 
Five-number summary, in distributions, 415 
Flesch-Kincaid Grade Level Score, 514-515 
Flesch Reading Ease Score, 514-515 
Flow aids, for presentations, 557-558 
Flow studies, 174 
Focused Interview, The (Merton), 160 
Focus groups 
attitude measurement scales from, 269 
discussion guide for, 153, 610-611 
executive, 37 
overview, 133 
population elements in, 340 
positioning research with, 484 
problems in, 160 
Followers, in focus groups, 160 
Food labeling, messages of, 206 
Forced-choice rating scales, 272-273 
Forced ranking measurement scale, 
285-286 
Forecasting, spreadsheet displays for, 421 
Formal studies, design of, 126 
Fractal-based transformation, 106 
Frame, sampling, 338, 347-348 
F ratio test statistic, for ANOVA, 
454-455, 488-489, 626-627 
Fraud, 99, 104 
Free association techniques, 161 
Freedom from bias, 258 
Free response measurement scale, 274 
Free-response questions, 308 
Frequency control, 360 
Frequency distributions, 398 
Frequency of response, 148 
Frequency tables, 407-408 
Friedman two-way ANOVA (analysis of 
variance), 462 
From the Headlines 
Anthony trial lessons for research oral 
presentations, 574 
Applications for fast response time 
technology, 397 
Charting of attitudes on tablet 
apps, 426 


Skyscraper correlation explanations, 
501 
Experiment to test United Airlines IT 
merger, 212 
Hearsay Social dashboards for social 
media, 47 
Hypotheses to explain aptitude test 
performance, 465 
Investment banking and scientific 
method, 72 
National Milk Producers Federation 
and obesity research proposal, 91 
Observation research at WalMart, 189 
Research hierarchy and Oreos, 120 
Oral presentation organization for 
Toyota research, 574 
Pepsi Live for Now campaign, 169 
Questions for iPad expectations 
questionnaire, 292 
Report development on small business 
optimism, 537 
Research design for renaming Kraft 
snack foods, 141 
Research design for moving IAMS pet 
food, 141 
Research measure to test efficiency of 
Walsworth Publishing, 264 
Sampling for Nike Foampoiste 
sneakers, 363 
Student research, 24 
Survey design for VOC data, 242 
Topical question development for 
government unemployment 
survey, 327 
“Frugging” (fund raising in guise of 
research), 236 
Functional area budgeting, 80 
Fund raising in guise of research 
(“frugging”), 236 
Funnel approach, for question 
sequencing, 319 
Fuzzy logic, 106 


G 


Galvanic skin response (GSR), 177 

Gamer type participants, 300 

Gamification, 83, 223 

Gamification (gamification methods), 83 

Gamification in Marketing and 
Gamification by Design 
(Zichermann), 223 

Gantt charts, 117, 599 

Garbology, 187 

General to specific question 
sequencing, 319 

Generation G, 223 

Generation Y, 145 

Genetic-based models, 107 

Genetic identification (DNA), 35 
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Geographic Information System 
(GIS), 418 

Geographs, 535 

Geometric means, as central tendency 
measures, 254 

GE Portfolio Matrix plotting, 421 

Gestures, 565 

GIS (Geographic Information 
System), 418 

Globalization, 6 

Global surveillance, 35 

Goodman and Kruskals’ gamma (Y) 
statistic, 495 

Goodness of fit testing, 487-490 

“Good research” characteristics, 15-18 

Government intervention, 6 

GPS (global positioning system), 405 

Graphic rating measurement scales, 
276, 284 

Green Book, a Guide for Buyers of 
Marketing Research Services 
(AMA Communications Services, 
Inc.), 100 

Grounded Theory, 158 

Group dynamics, 160 

Group interviews, 157-165 

Group time series research design, 210 

GSR (galvanic skin response), 177 

Gunning’s Fog Index, 514-515 

“Gut hunches,” value of, 65 
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Halo effect, 181, 274-275 
Handbooks, 99 
Harmonic means, as central tendency 
measures, 254 
Hawthorne studies, 64, 129 
Heterogeneous groups, 157 
High-speed Internet, 7 
Histograms, 408-410 
History, internal validity affected by, 
201, 210 
Homogeneous groups, 157 
Hostiles, in focus groups, 160 
“Hotspot,” 5 
Hunting of the Snark, The (Carroll), 551 
Hybrid Expectation measurement 
scales, 288 
Hyper-local initiatives, 175 
Hypotheses 
experimental, 195-196 
Hypothesis testing, 428-465 
bringing research to life vignette, 429 
experimental, 195-196 
logic of, 431-435 
overview, 58-61 
probability values (p values), 438-440 
research question in, 79 
significance tests, 440 
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Hypothesis testing—Cont. 
k-independent-samples, 453-460 
k-related-samples, 460-462 
one-sample, 444-447 
selecting, 442-443 
two-independent-samples, 447-450 
two-related-samples, 450-453 
types of, 440-442 

statistical significance, 430-431 

statistical testing procedures, 438 

Type I error, 435-436 

Type II error, 436-438 
Hypothetical constructs, 52 


IEDs (improvised explosive devices), 65 
Ignoramus type participants, 300 
Imaginary universe techniques, 155 
Imagination exercise techniques, 155 
Implantable computer chips, 60 
Impromptu speaking, 561, 562 
Improvised explosive devices (IEDs), 65 
Inaccessible households, telephone 
surveys and, 234 
Inaccurate numbers, telephone surveys 
and, 234-235 
Incentives, participation, 344 
Inconvenient Truth, An (Gore), 544 
Independent variables (IV) 
description of, 55-56 
in experiments, 192-193 
random assignment of groups and, 138 
Indexes, 98 
Indirect observation, 177 
Individual depth interviews, 12, 94, 
156-157 
Induction, 68—70 
Inferential level, for observation, 180 
Information overload, 6 
Information “snacking,” 6 
Informative changes, principle of, 557 
Informed consent of participants, 31—32 
In Search of Excellence (Peters and 
Waterman), 166 
Insider trading, 72 
Institutional Review Boards (IRBs), 
42-43 
Instrumentation, internal validity affected 
by, 202 
Instrument calibration logs, 85 
Instrument drafting, 315-324 
instructions, 320-323 
participant screening, 316-317 
pretesting, 324 
problems in, 323-324 
sequencing, 317-320 
Instrument error, 256 
Integrity in research, 28; see also Ethics 
Interaction extralinguistic behavior, 175 


Interaction of selection and X, external 
validity and, 203 
Interaction variables, 56—57 
Interactivity, 5 
Intercept interview, 238 
Intercept surveys, 226, 228 
Interim evaluation, 81 
Internal coalitions, of managers, 38 
Internal consistency, reliability as, 261 
Internal research proposals, 595 
Internal validity 
definition of, 257 
history problem in, 210 
threats to, 201—203 
Internet 
A/B tests for, 447 
analytics from, 13 
blogs, 97 
cloud computing, 103 
communities of practice on, 112 
data collection on, 35—36, 226 
dating services on, 194 
emerging research techniques on, 83 
feedback on, 394 
gaming on, 223 
high-speed, 7 
online focus groups, 164 
oral presentations on, 547 
performance reviews on, 145 
search engines, 100 
surveys on, 227, 273, 317-318 
workspace on, 4 
Interpretability in measurement, 262 
Interquartile range (IQR), 253, 401 
Interrater reliability, 261 
Interval estimates, in sampling, 365 
Interval scales of measurement 
attitude scales as, 253 
overview, 250 
samples and, 346 
statistical tests with, 443 
Intervening variables (IVVs), 57-58 
Intervention, 192; see also Experiments 
Interviews; see also Focus groups; 
Participants, research 
business-to-business, 37 
experts, 94 
group, 157-165 
individual depth, 12, 94, 156-157 
individual depth versus group, 152-153 
interviewer error, 220-221 
personal, for surveys, 237-238 
projective techniques in, 155-156 
responsibilities in, 153-155 
schedules for, 302 
telephone length limitations, 235 
telephone surveys, 232-237 
Intoxicated people, in focus groups, 160 
Intuition, 65 
Intuitive decision makers, | 1 


Inventory management, 174, 182 
Investigative questions, 79, 113-117; 

see also Management question 
IQR (interquartile range), 253, 401 
Item analysis, 278-279 


J 


Jargon, 564 
Jokers, in focus groups, 160 
Judgment sampling, 359 
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Kendall’s tau b (t,) statistic, 495-496 
Kendall’s tau c (7) statistic, 497 
Keyboarding data, 391-393 
Kinesthetic learning styles, 546 
Knowledge gap, in managers, 38 
Knowledge-intensive industries, 6 
Kolmogorov-Smirnov one-sample test, 623 
Kolmogorov-Smirnov two-samples test 
for large samples (two-tailed), 625 
Kruskal-Wallis test, 617-618 
k-samples nonparametric significance 
tests, 617-618; see also 
Nonparametric significance tests; 
Parametric significance tests 
Kurtosis, of distribution, 402 
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Laboratories, research, 379 
Laboratory conditions, for studies, 128 
Laboratory notes, 85 
Laddering techniques, 155 
Lambda (A) coefficient, 492-493 
Language of research, 50-64 
concepts, 50-52 
constructs, 52 
definitions, 52-55 
hypotheses, 58-61 
models, 63-64 
theory, 61-63 
variables, 55-58 
Learning styles, 545-546 
Least squares method, 482-485 
Leniency error, 272, 274 
Leptokurtic distribution, 402 
Letter of transmittal, of reports, 508 
Life histories, 158 
Likert measurement scales, 275, 
278-280, 288 
Linearity, of Pearson’s product moment 
coefficient r, 473 
Linear regression; see Simple linear 
regression 
Line graphs, 531-533 
Linguistic behavior, 175 
Listening tours, 13 


“Listening tour” research, 131 

Literature, research, 94, 596-597 

Location-based services (LBS), 30 

Logos, in persuasive communication, 
542-543 

Longitudinal studies, 128, 240 

Loyalty identification programs, 102 


Magnetic resonance imaging (MRI), 177 
Management dilemma, 6, 20, 77, 79 
Management question 
in actionable research development, 
108-112 
hierarchy of, 296-301 
communication approach, 298-299 
disguising objectives and sponsors, 
299-300 
preliminary analysis, 300-301 
scale needed, 297-298 
instrument drafting, 315-324 
instructions, 320-323 
participant screening, 316-317 
problems in, 323-324 
sequencing, 317-320 
measurement question construction, 
302-315 
categories and structure, 302 
content, 302—304 
effective, 328-334 
response strategy, 306-312 
sources of, 312-315 
wording, 304-306 
pretesting, 324 
Management reports, 506-507; see also 
Reports, research 
Management-research question 
hierarchy, 77 
Manipulating independent variables (IV), 
in experiments, 193 
Mann-Whitney test, critical values of U 
(table), 628 
Manual for Writers of Term Papers, 
Theses, and Dissertations, A 
(Turabian), 514 
Manuscript reading, 562 
Mapping, 246, 250, 418-419 
Marketing research online communities 
(MROCs), 144 
Matching, in experiments, 199 
Maturation, internal validity affected 
by, 201 
Maximum-flow model, 63 
McNemar test, 452-453, 462 
Mean 
calculation of, 400 
as central tendency measure, 254 
population mean estimation, 365-368 
sample size and, 368-370 


Mean square, for F ratio, 454 
Measurement, 244-265 
bringing research to life vignette, 
245-246 
concepts in, 51 
differences in, 254-256 
of objects and properties, 248—249 
overview, 246-248 
practicality of, 262 
reliability of, 260-261 
scales for, 249-254 
interval, 253 
nominal, 250-252 
ordinal, 252 
ratio, 253-254 
validity of, 257-260 
Measurement question, 302—315 
in actionable research development, 
118 
categories and structure, 302 
content, 302-304 
effective, 328-334 
in management question hierarchy, 79 
response strategy, 306-312 
sources of, 312-315 
wording, 304-306 
Measurement scales, 266-293 
attitude, 51, 268-271 
bringing research to life vignette, 267 
constant-sum, 284 
cumulative, 289-290 
graphic rating, 284 
Likert, 278-280 
management question hierarchy and, 
297-298 
numerical/multiple rating list, 283-284 
ranking, 285-287 
selecting, 271-275 
semantic differential, 280-283 
simple attitude, 275-277 
sorting and, 287-289 
Stapel, 284 
Measures of association; see Association, 
measures of 
Measures of central tendency; see Central 
tendency, measures of 
Media consumption patterns, 419 
Median measure of central tendency 
in boxplots, 415-416 
calculation of, 400-401 
overview, 253 
Megatrends (Naisbitt Group), 385 
Memorization, 562 
Memory decay, 329 
Mesokurtic distribution, 402 
Message power, 206 
Metaphor, in presentations, 553 
Metaphor elicitation techniques, 156 
Method of agreement, 136 
Mining internal sources, 88, 102-107 


685 


>subject index 


Missing at random (MAR) data, 391 
Missing but not missing at random 
(NMAR) data, 391 
Missing completely at random (MCAR) 
data, 391 
Missing data, coding, 389-391 
Mixed-access sampling, 346 
Mixed-mode research, 229 
MLA Handbook for Writers of Research 
Papers (Gibaldi), 514 
Mobile devices; see also Smart phones 
questionnaire design for, 303 
research on, 232 
surveys on, 33, 83, 237 
Mobile ethnography techniques, 83 
Mobile qualitative techniques, 83 
Mobile surveys, 303, 344 
Models, research, 63-64 
Mode measure of central tendency, 252 
Moderating variables (MV), 56-57 
Modular report writing, 509 
Monitoring, data collection by, 127 
Monotonic trends, in population 
elements, 352 
Moral choices, ethics for, 28 
Mortality, internal validity affected by, 
202-203 
Most Human Human: What Artificial 
Intelligence Teaches Us about 
Being Alive, The (Christian), 447 
Motivated sequence organization, of 
presentations, 549 
MRI (magnetic resonance imaging), 177 
MROCs (marketing research online 
communities), 144 
Multidimensional measurement 
scales, 272 
Multimethod phone studies, 232 
Multiphase sampling, 357 
Multiple-choice, multiple-response 
measurement scale, 277 
Multiple-choice, single-response 
measurement scale, 277 
Multiple choice measurement scale, 
274-275 
Multiple choice questions, 307-311 
Multiple comparison tests, 
457-458 
Multiple group time series quasi- 
experiment, 208 
Mutual exclusivity, in coding, 383 
“Mystery shopper” scenario, 129, 165 


Narrative organization of oral 
presentations, 549-551 
Netnography, 144, 394 
Neural networks, 107 
Neuroscience, 65 
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Neuroimaging techniques, 177 
New-product and service design, 
spreadsheet displays for, 421 
Nominal measurement scales 
data display of, 408 
description of, 250-252 
nonparametric measures of association 
for, 490-494 
samples and, 346 
statistical tests with, 443 
Nonbehavioral observations, 173-175 
Noncompletion error, in surveys, 229 
Noncontact rate, in telephone 
surveys, 233 
Noncoverage error, 346 
Nondirectional test, in hypothesis 
testing, 432 
Nondisclosure, 34, 36-37 
Nonequivalent control group design, 209 
Nonexpert groups, 157 
Nonfunctioning numbers, telephone 
surveys and, 234-235 
Nonparametric measures of association 
for nominal data, 490-494 
for ordinal data, 494-498 
Nonparametric significance tests, 440 
chi-square (x7) test, 445-446 
in k-independent-samples situation, 
460 
in k-related-samples situation, 462 
k-samples, 617-618 
one-sample, 612-613 
in one-sample situation, 445 
in two-independent-samples situation, 
448-450 
in two-related-samples situation, 
452-453 
two-samples, 613-617 
Nonparametric statistics, 252 
Nonprobability sampling 
convenience, 359 
description of, 343 
practical considerations, 358-359 
purposive, 359-360 
in qualitative research, 152 
quota, 199 
snowball, 360-361 
Nonresistant statistics, 416 
Nonresponse error, 346 
Nonverbal communication, 175, 
564-565 
Normal distribution, 398, 473, 620 
Normative models, 63 
Null hypothesis, 432-433, 438; see also 
Hypothesis testing 
Numerical measurement scale, 276 
Numerical/multiple rating list 
measurement scales, 283-284, 
288 
Nuremberg Code, 42 
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Obesity, 131 
Objects, variables as, 248-249 
Observations, 170-189 
behavioral, 175 
bringing research to life vignette, 171 
concordant and discordant, 495 
conducting 
content specification, 180 
data collection, 181-184 
observer training, 181 
study type, 178-180 
data collection from, 85 
evaluation by, 69 
evaluations of, 176 
in experiments, 206 
nonbehavioral, 173-175 
observer-participant relationship, 
176-178 
overview, 172-173 
retail behavior, 486 
sources and locations for, 12 
unobtrusive measures for, 184-187 
OCR (optical character recognition), 393 
Office romance, survey on, 319 
OMR (optimal mark recognition), 393 
One-group pretest-postest design, 
204-205 
One-sample nonparametric significance 
tests, 612-613 
One-tailed test, in hypothesis testing, 432 
One-way ANOVA (analysis of variance), 
453-457 
Online Communities, 83, 112, 164 
Online dating services, 194 
Online focus groups, 164 
Online research, 5 
Online surveys, 273 
Open-ended questions 
coding, 382 
content analysis for, 384-387 
responses to, 306, 308 
Open rates, for e-mail, 198 
Operational definitions, 53-55, 249 
Operationalized hypotheses, 195 
Opinion mining, 105 
“Oprah factor,” 188 
Optical recognition, data entry by, 
393-394 
Optimal mark recognition (OMR), 
393, 404 
Option analysis, 81 
Oral history, 158 
Oral presentations, 538-574; see also 
Reports, research 
Aristotle’s three persuasive 
communication principles, 541-543 
bringing research to life vignette, 539 
delivering, 562-565 


model for, 540-541 
organizing 
motivated sequence, 549 
narrative, 549-551 
three-point speech, 551 
traditional, 548-549 
planning 
audience analysis, 544-545 
learning styles, 545-546 
recall accuracy, 546-547 
Web-delivered, 547 
practicing and arranging, 566-570 
supporting materials, 551-554 
visualizing, 554-561 
design principles, 557-561 
psychological and physical 
foundations, 555—557 
slide improvement, 561 
Ordinal measurement scales 
nonparametric measures of association 
for, 494-498 
overview, 250 
samples and, 346 
statistical tests with, 443 
uses of, 252 
Oscar awards, movie viewership and, 478 
Outliers, 415-416 
Outliers (Gladwell), 566 
Outlining reports, 513 
Outsourcing survey services, 239-240 
Oxford English Dictionary, 187 
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Pace, in reports, 516 
Paired-comparison measurement scale, 
285-287 
Panel studies, 128, 240, 343-344 
Paradoxes techniques, 155 
Paralanguage, 565 
Parallel forms of tests, for reliability, 261 
Parametric significance tests 
in k-independent-samples situation, 
453-460 
multiple comparison tests, 457-458 
one-way ANOVA (analysis of 
variance), 453-457 
a priori contrasts, 457 
two-way ANOVA (analysis of 
variance), 458-460 
in k-related-samples situation, 460-462 
in one-sample situation, 444-445 
overview, 440 
in two-independent-samples situation, 
447-448 
in two-related-samples situation, 
451-452 
Parametric statistics, 252-253 
Pareto diagrams, 415 
Participant observation, 178 


Participants, research 
ethical issues, 28-36 
benefits of research discussed, 29-30 
debriefing, 32 
deception, 30-31 
informed consent, 31-32 
Internet data collection, 35—36 
privacy rights, 32-34 
for experiments, 199 
honesty of, 300 
incentives for, 344 
internal validity and selection of, 202 
knowledge of, 329 
in observations, 178 
observer-participant relationship, 
176-178 
perceptual awareness of, 129 
physicians and patients as, 184 
qualitative, 148 
rapport with, 323 
recruiting, 33, 153 
screening, 316-317 
in self-administered surveys, 230 
survey errors from, 221-222, 224 
in telephone surveys, 236 
Pathos, in persuasive communication, 
542, 544 
Pattern discovery, data mining for, 104-105 
Pattern thinking, 12 
Peacock type participants, 300 
Pearson’s product moment coefficient r, 
469-470, 473-476 
Penetration pricing, 62 
People intelligence, 255 
Percentages, in cross-tabulation, 420-423 
Perceptual awareness of participants, 129 
Perceptual organization, principle of, 557 
Performance anxiety, 567-568 
Performance review research, 145 
Personal interviews, 225, 237-238 
Personification techniques, 155 
Persuasive communication principles, 
541-543 
PET (positron emission tomography) 
scans, 177 
Photographic framing, for presentations, 
558 
Physical condition analysis, 174 
Physical traces, indirect observation 
based on, 184, 187 
PicProfile feature 
emerging research techniques, 83 
focus groups, 269 
graphics, 555 
Internet data collection, 226 
mixed-access sampling, 346 
mixed-mode research, 229 
observation, 175 
population-representative survey 
methodology, 311 


product idea sources, 162 
projective techniques, 150 
qualitative and quantitative research 
for advertising, 484 
questionnaires, 309 
research methodology 
developments, 17 
social media, 277 
survey length, 318 
survey measurement scales, 273 
unstructured information, 384 
Web surveys, 317 
Pictographs, 535 
Picture association techniques, 155 
Picture supremacy, for presentations, 558 
Pie charts, 407-408, 534 
Pilot testing, 85, 199 
Planning research design, 15-16 
Playkurtic distribution, 402 
Pleaser type participants, 300 
Point estimates, in sampling, 364-365 
Politically motivated research, 54, 89 
Polling, 86 
Population, 84, 338 
Population elements, 338-339, 
343-344, 352 
Population mean, estimation of, 
365-368 
Population parameters, 345 
Population proportion of incidence, 346 
Population-representative survey 
methodology, 311 
Positioning research, 421, 484 
Positron emission tomography (PET) 
scans, 177 
Posttest-only control group research 
design, 207 
Posture, 565 
Practicality, measurement, 262 
Precision, 328, 341 
Precision control, 360 
Precoding, 381 
Predesigned measurement 
questions, 118 
Prediction, data mining for, 104 
Prediction Markets, 83 
Predictive models, 63 
Predictive studies, 22 
Predictive validity, 259 
Predictor variables, 55 
Preexperimental design, 204-206 
Prefatory items, in reports, 508 
Presentation Zen (Reynolds), 543 
Presenting Numbers, Tables, and Charts 
(Bigwood and Spore), 412 
PRE (proportional reduction in error) 
statistics, 492-493, 495 
Pretasking, in qualitative research, 
148, 150 
Pretesting research, 85, 199, 309, 324 
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Pretest-posttest research designs 
control group design, 207 
one-group, 204-205 
posttest-only control group design, 207 
separate sample, 209-210 

Pricing, penetration, 62 

Primacy effect, 310, 546 

Primary data, 86, 130 

Primary demand, in product life 

cycle, 62 

Primary sources of information, 96 

Prior evaluation, 81 

Privacy Act of 1974, 42 

Privacy Protection Act of 1980, 42 

Privacy rights of participants 
on Internet, 35—36 
location-based services and, 30 
mail versus online surveys, 229 
overview, 32-34 
sentiment analysis and, 105 

Probability sampling 
complex, 352-358 
description of, 343 
simple random, 349-351 

Probability values (p values), 438-440 

Problem-solving, as research basis, 15 

Process analysis, 174 

Product idea sources, 162 

Product life cycle, 62 

Professional peer communities, 112 

Professional standards of ethics, 40-42 

Projective techniques, 150, 155-156 

Proofreading reports, 516-517 

Properties, variables as, 248-249 

Property-behavior asymmetrical 

relationships, 138 

Property-disposition asymmetrical 

relationships, 138 
Proportional reduction in error (PRE) 
statistics, 492-493, 495 
Proportionate stratified sampling, 
354-355 
Proportions, 346, 370-371 
Proposal, research, 585-610; see also 
Reports, research 
checklist for, 591 
evaluating, 601-605 
example of, 114-117, 606-609 
overview, 80-82 
request for proposal (RFP), 3, 7, 
586-590 
researcher benefits of, 593 
sections of, 595-601 
sponsor of, 592 
types of, 593-595 
Proposal reviews, 3-4, 601-605 
Propositional units, in content analysis, 
385 

Propositions, 58; see also Hypothesis 

testing 
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Proselytizers, in focus groups, 160 

Proxemics, 175 

Public speaking, fear of, 567 

Purchase decisions, 51, 62, 300 

Pure research, 15 

Purpose nondisclosure, 37 

Purposive nonprobability sampling, 
359-360 

Purposive sampling, 152 

Puzzle-solving activity, scientific 
method as, 66 
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Q-sorts, 287 
Qualitative research, 142-169 
on advertising, 484 
bringing research to life vignette, 143 
description of, 144 
methodologies, 123, 151-167 
combining, 165-166 
Readability interviews, 157-165 
individual depth interviews, 156-157 
interviews, 152-156 
sampling, 151-152 
nonprobability sampling in, 152 
process, 148-151 
quantitative research merged with, 
166-167 
quantitative research versus, 144-148 
Quantitative research 
on advertising, 484 
qualitative research merged with, 
166-167 
qualitative research versus, 144-148 
Quartile deviation, 401-402 
Quartiles, as resistant statistics, 415-416 
Quasi-experimental designs, 207-210 
Questionnaires, 85, 179-180, 309; see also 
Management question; Surveys 
QuickQuery online survey (Harris 
Interactive), 319 
Quota matrix, 199-200 
Quota sampling, 359 


Radio frequency identification (RFID), 
60, 418 
Random assignment to groups, 138, 199 
Random dialing, telephone surveys 
and, 234 
Random numbers, statistical table of, 629 
Random sampling error, 341 
Range, 401 
Ranking measurement scales 
characteristics of, 274 
questions for, 307, 312 
response type for, 271 
uses of, 285-287 


Rater error, 274—275 
Rating measurement scales 
characteristics of, 274 
questions for, 307, 311-312 
response types for, 271-273 
Ratio measurement scales 
overview, 250 
samples and, 346 
statistical tests with, 443 
uses of, 253-254 
Reactivity of testing on X, external 
validity and, 203 
Reactivity response, 184 
Readabiity index, 514 
Real-time access, 7 
Recall accuracy, 546-547 
Recency effect, 310, 546 
Reciprocal relationships of 
variables, 138 
Record analysis, in observation, 173 
Recording group interviews, 165-166 
Recruitment screeners, 153-154 
Referential units, in content analysis, 385 
Refusal rate, in telephone surveys, 233 
Regression coefficients, 479 
Relational hypotheses, 59 
Relationships, for presentations, 559 
Relevance, principle of, 555 
Relevant criteria, 258 
Reliability, 181, 259-261; see also Validity 
Reliable criteria, 258 
Repeated measures ANOVA (analysis of 
variance), 460-462 
Repeat purchasers, 62 
Replicable research, 15, 193 
Reports, research, 502-537; see also Oral 
presentations 
bringing research to life vignette, 503 
components of, 507-512 
drafts, 514-517 
of group interviews, 165-166 
long, 505-507 
overview, 21 
presentation considerations, 517 
prewriting concerns, 512-514 
purpose of, 127 
on results, 86-87 
short, 504-505 
statistical presentation, 517 
example of, 518-528 
graphics for, 529-536 
semitabular, 529 
in text, 517 
Representative samples, 204, 343 
Request for proposal (RFP), 3, 7, 
586-590; see also Proposal, 
research 
Rescaling variables, 254 
Research-as-expense-not-investment 
mentality, 88 


Research design, 121-141 
bringing research to life vignette, 
123-124 
causal studies, 136-139 
description of, 124-129 
descriptive studies, 134-136 
for experiments, 197-198 
exploratory studies, 129-134 
overview, 82-85 
preexperimental, 204-206 
quasi-experimental, 207-210 
semi-experimental, 207-210 
true experimental, 206-207 
Researcher teams, 37-40 
Research findings, unambiguous 
presentation of, 17-18; see also 
Reports, research 
Research methodologies, 7 
Research objectives, 271, 299-300 
Research process, 74-91 
bringing research to life vignette, 75 
company database strip-mining, 88 
data analysis and interpretation, 86 
data collection, 85-86 
design, 82-85 
favored-technique syndrome, 87-88 
ill-defined problems, 88-89 
overview, 13-15 
politically motivated, 89 
proposals, 80-82 
questions, 77-79 
reporting, 86-87 
sequence of, 76-77 
unresearchable questions, 88 
Research proposal; see Proposal, research 
Research question; see also Management 
question 
in actionable research development, 
112-113 
crystallization of, 126 
ill-defined, 88-89 
overview, 77-79 
Research question, secondary data to 
clarify 
actionable research development, 
108-118 
investigative questions, 113-117 
management question, 108-112 
measurement questions, 118 
research question, 112—113 
bringing research to life vignette, 93 
exploration, 94—102 
evaluating information sources, 
100-102 
information levels, 96-97 
information sources, 97-100 
mining internal sources, 102—107 
Research variables, 55—58 
control of, 127 
decision, 82 


description of, 21 
in experiments, 195 
measurement of, 248—249 
prioritizing, 60 
relationships of, 138 
rescaling, 254 
Residuals, standardized, 485 
Resistant statistics, 415 
Resource allocation, 80-82, 421 
Response error, 222—224, 256 
Response frequency, 148 
Response strategy 
alternative, 324 
measurement question construction 
and, 305-312 
checklist, 311 
dichotomous, 308 
free-response, 308 
multiple choice, 308-311 
ranking, 312 
rating, 311-312 
question choice for, 332 
scales for, 271-272 
Retail shopping patterns, 486 
Retention, of customers, 242 
Retinal scans, 35, 60 
Return on investment (ROD), 6 
RFID (radio frequency identification), 
60, 418 
RFP (request for proposal), 3, 7, 586-590; 
see also Proposal, research 
Right to privacy, 34; see also Privacy 
rights of participants 
Right to quality research, 38-39 
Right to safety, 39 
Risk, 30, 104 
Role playing techniques, 161 
Romance in office, survey on, 319 
RSS aggregators, 97 
Rule of three, 551 
Rule-of-thumb budgeting, 80 


Ss 


SaaS (software-as-a service) products, 107 
Safety compliance, 174 
Salience, principle of, 557 
Sample statistics, 345 
Sampling, 336-372 
accuracy of, 340-341 
bringing research to life vignette, 
337-338 
in data mining, 106—107 
designing, 341-344 
design of, 84-85 
for experiments, 199 
frame for, 347-348 
interest parameters, 345-346 
methods for, 348 
mixed access, 346 


nonprobability 
convenience, 359 
practical considerations, 358-359 
purposive, 359-360 
snowball, 360-361 
precision of, 341 
probability 
complex, 352-358 
simple random, 349-351 
purpose of, 338-339 
in qualitative research, 151-152 
representative, 204 
size of sample, 364-371 
interval estimates, 365 
overview, 348-349 
point estimates, 364-365 
population mean estimation, 
365-368 
questions on means and, 368-370 
questions on proportions and, 
370-371 
survey accessibility of, 228 
target population, 345 
Sampling error, 220 
Scales; see Measurement scales 
Scaling, 254, 270-271 
Scalogram analysis, 289-290 
Scanning, optical, 393 
Scatterplots, 470-473, 483 
Scheme, coding, 380 
Scientific definitions, 54 
Scientific method, 64-70 
deduction, 66-70 
“gut hunches,” 65 
induction, 68—70 
overview, 15 
SCNT (somatic cell nuclear transfer), 54 
Screeners, recruitment, 153-154 
Search engines, 97, 100, 350-351 
Secondary data, 86, 130 
Secondary sources of information, 96 
Segmentation/targeting analysis, 
spreadsheet displays for, 421 
Self-administered surveys, 225-232 
Self-delusionist type participants, 300 
Selling in guise of research (“sugging’”’), 
236 
Semantic differential (SD) measurement 
scales, 276, 280-283 
Semantic mapping techniques, 156 
Semi-experimental designs, 207-210 
Semi-structured interviews, 153 
Semitabular presentation of statistics, 529 
Sensitive information, 330 
Sensory sort techniques, 155 
Sentence completion techniques, 155 
Sentence outline, for reports, 513 
Sentiment analysis, 105 
Separate sample pretest-posttest design, 
209-210 
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Sequence-based models, 107 
Sequencing questions, 317—320 
Sequential interviewing, 158 
Sequential sampling, 357 
Serial position effect, 546 
Shape measures, statistics for, 402 
Shopping patterns, 486 
Significance tests 
k-independent-samples, 453-460 
k-related-samples, 460-462 
nonparametric, 612-618 
k-samples, 617-618 
one-sample, 612-613 
two-samples, 613-617 
one-sample, 444-447 
for Pearson’s product moment 
coefficient r, 475-476 
selecting, 442-443 
two-independent-samples, 447-450 
two-related-samples, 450-453 
types of, 440-442 
Simple attitude measurement scales, 
275-277 
Simple category scale, 275 
Simple linear regression, 479-490 
application of, 480-482 
goodness of fit testing, 487-490 
least squares method, 482-485 
model of, 479-480 
predictions from, 486-487 
Simple observation, 178 
Simple random probability sampling, 
349-351, 357 
Simplicity, for presentations, 559-560 
Simulations, 128 
Situational factors, error and, 256 
Skimming, high pricing and, 62 
Skip interval, in systematic 
sampling, 352 
Slope (8), as regression coefficient, 
479-480 
Smart cards, 35-36 
Smart phones; see also Mobile devices 
location-based services and, 30 
purchase decisions affected by, 51 
questionnaire design for, 303 
research on, 232 
surveys on, 237 
“Snacking,” on information, 6 
Snapshot feature 
advanced statistics, 494 
blogs, 97 
brain scans, 177 
business intelligence, 99 
business research, 5 
cloud computing, 103 
communities of practice, 112 
consultancy skills, 10 
customer satisfaction measurement, 
280 
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Snapshot feature—Cont. 
data mining, 105 
data visualization, 410 
dating services, 194 
e-mail open rates, 198 
e-speed, 506 
ethics of mobile surveys, 33 
eye-tracking studies, 183 
feedback generation, 79 
focus groups, 160 
food labeling, 206 
gamification, 223 
“gut hunches,” 65 
high-touch strategy, 157 
hypothesis testing, 434, 442 
incentives, 344 
“listening tour” research, 131 
location-based services, 30 
media consumption patterns, 419 
mobile questionnaires, 303 
modular report writing, 509 
“mystery shopping,” 165 
netnography, 394 
office romance, 319 
oral presentations, 544 
Oscar awards, 478 
paired-comparison measurement 
scale, 287 
pattern thinking, 12 
performance review research, 145 
physicians and patients as research 
subjects, 184 
privacy, 34 
public speaking, fear of, 567 
radio frequency identification 
(RFID), 182 
representative samples, 204 
research laboratories, 379 
research variables, 60 
retail shopping patterns, 486 
RFID (radio frequency identification), 
182 
sampling, 340 
scientific definitions and politics, 54 
search engines, 100 
skipped and watched advertisements, 254 
smartphone-based research, 232 
split-run testing online, 447 
spreadsheets, 421 
statistics understanding, 422 
talent analytics, 255 
telephone surveys, 237 
text analytics, 107 
videoconference focus groups, 163 
video insights, 84 
Snowball sampling, 152, 360-361 
Social desirability bias, 224 
Social-desirability error, 229 
Social Media Analysis, 144 


Social media 
location-based services for, 30 
monitoring, 107 
netnography on, 144 
product information from, 277 
Social Networking, 105, 145, 194, 319, 
407-408 
Software-as-a-Service (SaaS) products, 107 
Solicited research proposals, 595 
Somatic cell nuclear transfer (SCNT), 54 
Somer’s d statistic, 497 
Sorting, 272, 287-289 
Sound reasoning, 64, 66-70 
Spatial relationships, 175 
Speaker note cards, 563 
Spearman-Brown correction formula, 261 
Spearman’s rho (p) correlation, 497 
Specific instances, in presentations, 552 
Speech recognition functionality, 394 
Split-ballot technique, 310 
Split-half technique, 261 
Split-run testing online, 447 
Split-sample tests, 280 
Sponsors, research 
background of, 49-50 
disguising, 299-300 
ethical issues, 36-39 
focus group viewing by, 160 
Spreadsheets, 392-393, 421 
Stability, reliability as, 260-261 
Stage fright, 567-568 
Standard deviation, 401; see also 
ANOVA (analysis of variance) 
Standard global time, 253 
Standard Industrial Classifications, 99 
Standardized decision makers, 11 
Standardized tests, 85 
Standard normal distribution, 620 
Standard scores (z-scores), 398 
Stapel measurement scales, 276, 284 
State of the Blogosphere Report 
(Technorati), 97 
Static group comparison research design, 
205-206 
Statistical power, 252 
Statistical regression, 202 
Statistical significance, 430-431; see also 
Significance tests 
Statistical studies, 128 
Statistical tables, 619-629 
areas of the standard normal 
distribution, 620 
critical values of D in the Kolmogorov- 
Smirnov one-sample test, 623 
critical values of D in the Kolmogorov- 
Smirnov two-samples test for 
large samples (two-tailed), 625 
critical values of t for given probability 
levels, 621 


critical values of the chi-square 
distribution, 622 
critical values of the F distribution for 
a = .05, 626-627 
critical values of Tin the Wilcoxon 
matched-pairs test, 623 
critical values of U in Mann-Whitney 
test, 628 
random numbers, 629 
Statistics, 398-402; see also Hypothesis 
testing; Significance tests 
central tendency measures, 400-401 
distributions, 398-400 
presentation of 
example of, 518-528 
graphics for, 531-536 
semitabular, 529 
tabular, 529-530 
in text, 517 
in presentations, 552 
shape measures, 402 
value of, 422 
variability measures, 401-402 
Stem-and-leaf displays, 411-414 
Stem-cell research, 54 
Stimulus-response asymmetrical 
relationships, 138 
Store audits, 174 
Storytelling, in presentations, 550-551, 
553-554 
Strategy, 9-11 
Stratified random sampling, 353 
Stratified sampling, 353-355, 357 
Stratum charts, 534 
Strip-mining company database, 88 
Structured interviews, 153 
Structured response, 306 
Structured sorts, 287 
“Sugging” (selling in guise of research), 
236 
Summated rating scales, 278 
Surface charts, 534 
Surveys, 214-242; see also Management 
question 
bringing research to life vignette, 
215-216 
communication approach, 216-226 
choosing, 224-226 
description of, 216-219 
errors in, 219-224 
experience, 131-132 
gamification, 83, 223 
incentives, 225, 230, 231, 273, 
344, 394 
length of, 318 
measurement scales for, 273 
mixed-mode, 229, 240 
mobile, 33, 83, 303, 344 
mobile devices and, 33 


optimal method for, 238-240 
personal interview, 237-238 
population-representative survey 
methodology, 311 
self-administered, 226-232 
telephone interview, 232—237 
Web, 317-318 
wildcat, 134 
Sustainability, 131 
Syllogism, in appeals, 542 
Symmetrical relationships of 
variables, 138 
Syntactical units, in content 
analysis, 385 
Systematic observation, 178 
Systematic sampling, 199, 352-353, 357 
Systematic variance, 340-341 


T 


Tables 
analysis based on, 423-425 
data presented in, 412-414 
statistics presentation in, 529-530 
Tablets, questionnaire design for, 303 
Tactics, 10 
Talent, analytical, 6 
Talent analytics, 255 
Target customers, 242 
Target marketing, 104 
Target population, 84, 345 
Target questions, 302 
Task budgeting, 80 
TDE (touch-tone data entry), 233 
TDM (Total Design Method), 230 
Technical reports, 506 
Telemarketing, 235-236 
Telephone focus groups, 163-164 
Telephone interviews, 225, 232-237 
Telephone surveys, 309 
Temporal extralinguistic behavior, 175 
10-minute rule, 546 
Tertiary sources of information, 97 
Testimony, in presentations, 553 
Testing, 85, 202 
Test markets, 148 
Text analytics, 107 
Text-box responses, on cell phones, 232 
Thematic Apperception Test, 155 
Thematic units, in content analysis, 385 
Theory, 61-63 
Thick description, in case studies, 166 
Thinking, research, 48-73 
bringing research to life vignette, 
49-50 
language, 50-64 
concepts, 50-52 
constructs, 52 
definitions, 52-55 


hypotheses, 58-61 
models, 63-64 
theory, 61-63 
variables, 55-58 
scientific method, 64-70 
deduction, 66—70 
“gut hunches,” 65 
induction, 68—70 
“Threat assessment” abilities, 65 
3-D graphics, 535-536 
360-degree formal feedback 
systems, 145 
Three-point speech, 551 
Time constraints in surveys, 228 
Time dimension, in studies, 128 
Time/motion studies, 174 
Time order of variables, 137-138 
Time sampling, in observations, 181 
Title page, of reports, 508 
Tone, in reports, 516 
Topical scope, of studies, 128 
Top-line growth, 242 
Total Design Method (TDM), 230 
Total quality management (TQM), 87 
Touch-tone data entry (TDE), 233 
TQM (total quality management), 87 
Tracking capabilities, 6 
Traditional organization, of presentations, 
548-549 
Training observers, 181 
Transactional information, 12 
Transaction data, 102 
Treatment levels, in experiments, 
195-197 
Trends, data visualization for, 106 
Triangulation, 166, 187 
True experimental designs, 206-207 
t-Test, 488, 621 
Two-independent-samples significance 
tests, 447-450 
Two-related-samples significance tests, 
450-453 
Two-samples nonparametric significance 
tests, 613-617 
Two-stage design, 133 
Two-tailed test, in hypothesis 
testing, 432 
Two-way ANOVA (analysis of 
variance), 458-460 
Type I error, in hypothesis testing, 
435-436 
Type II error, in hypothesis testing, 
435-438 


U 


Unbalanced rating scales, 272 
Unconscious decision maker type 
participants, 300 
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Unforced-choice rating scales, 

272-273 

Unidimensional measurement 
scales, 272 

niversal Product Code (UPC), 394 

nobtrusive measures, for observation, 
181, 184 

nsolicited research proposals, 595 

nstructured information, 384 

nstructured interviews, 153 

nstructured response, 306 

nstructured sorts, 287 

.S. Safe Harbor Agreement, 35 


U 
U 
U 
U 
U 
U 


Vv 


Validity; see also Reliability 
of experiments, 201-204 
measurement, 257-260 
observer drift to reduce, 181 
Variability, statistics for, 401-402 
Variables; see Research variables 
Variance, 340-341, 401; see also 
ANOVA (analysis of variance) 
Verbal stylistic extralinguistic 
behavior, 175 
Verification of data, 85 
Videoconferencing focus groups, 164 
Video insights, in research, 84 
VideoMarker technology (Focus Vision), 
163 
Virtual communities, 112 
Virtual Environments, 83 
Virtual groups, 144 
Visibility, of presentations, 557 
Visionaries, decision makers as, | 1 
Visitor from another planet techniques, 
155 
Visualization tools, 7 
Visual Analysis 106, 183, 410, 411, 471, 
545, 547, 549, 554-561 
Visual learning styles, 545 
Visual preparation, for presentations, 
557 
Vocabulary, 304, 330-331 
Vocal extralinguistic behavior, 175 
Voicemail option, surveys and, 237 
Voice-of-consumer (VoC) data, 242 
Voice recognition (VR), 233, 394 


WwW 


Wallflowers, in focus groups, 160 
Webcam-based Interviews, 83 
Whitespace, for presentations, 558 
Why Most PowerPoint Presentations 

Suck (Altman), 554 
Wilcoxon matched-pairs test, 623 
Wildcat survey, 134 
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Word association techniques, 155 
Wording, of measurement questions, 
304-306, 330-333 
Workspace, on Internet, 4 
World Wide Web 
A/B tests for, 447 
evaluating web sites, 101 


feedback on, 394 

oral presentations on, 547 
survey research on, 227, 231 
surveys on, 317-318 
workspace on, 4 


xX 
XSight software (QSR International), 384 


Zz 


z-scores, 398 
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